[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:30:39 dist-test-slave-8w9v krb5kdc[9065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:30:39 dist-test-slave-8w9v krb5kdc[9065](info): set up 2 sockets
Aug 07 19:30:39 dist-test-slave-8w9v krb5kdc[9065](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 19:30:40.420814 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.005s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:30:40 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:30:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:30:40.620049 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45547
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:32787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:40.987350 9079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:40.987684 9079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:40.987778 9079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:40.995775 9079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:30:40.995914 9079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:40.995985 9079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:40.996086 9079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:30:40.996207 9079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:30:41.007402 9079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:32787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:41.008925 9079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:41.010407 9079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:41.024842 9088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:41.024873 9086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:41.024673 9085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:41.025502 9079 server_base.cc:1034] running on GCE node
I20260807 19:30:41.026465 9079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:41.028026 9079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:41.029237 9079 hybrid_clock.cc:648] HybridClock initialized: now 1786131041029197 us; error 84 us; skew 500 ppm
I20260807 19:30:41.029654 9079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:41 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:41.037758 9079 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:30:41.039525 9079 webserver.cc:460] Webserver started at http://127.8.216.190:43147/ using document root <none> and password file <none>
I20260807 19:30:41.040324 9079 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:41.040544 9079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:41.040879 9079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:41.044821 9079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "3130f7d5587340e08f31403501075673"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "ba4a8b79ed8642fc2b8365896850e832"
server_key_iv: "5322179d521167ae7739caefa42e77d6"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.045792 9079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "3130f7d5587340e08f31403501075673"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "ba4a8b79ed8642fc2b8365896850e832"
server_key_iv: "5322179d521167ae7739caefa42e77d6"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.052873 9079 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260807 19:30:41.057616 9096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:41.059180 9079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:41.059414 9079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "3130f7d5587340e08f31403501075673"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "ba4a8b79ed8642fc2b8365896850e832"
server_key_iv: "5322179d521167ae7739caefa42e77d6"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.059626 9079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:41.085232 9079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:41.090175 9079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:41.090523 9079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:41.114447 9079 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:45547
I20260807 19:30:41.114552 9150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:45547 every 8 connection(s)
I20260807 19:30:41.116276 9079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:30:41.120944 9151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:41.121528 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9079
I20260807 19:30:41.121913 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:30:41.124697 9058 external_mini_cluster.cc:1428] Setting key 9060a153c7ac68d601a94fa3427ac218
I20260807 19:30:41.134766 9151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673: Bootstrap starting.
I20260807 19:30:41.138019 9151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:41.139338 9151 log.cc:824] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:41.142647 9151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673: No bootstrap required, opened a new log
I20260807 19:30:41.151027 9151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } }
I20260807 19:30:41.151432 9151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:41.151556 9151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3130f7d5587340e08f31403501075673, State: Initialized, Role: FOLLOWER
I20260807 19:30:41.152225 9151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } }
I20260807 19:30:41.152535 9151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:30:41.152685 9151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:30:41.152872 9151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:41.155411 9151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } }
I20260807 19:30:41.156050 9151 leader_election.cc:302] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3130f7d5587340e08f31403501075673; no voters:
Aug 07 19:30:41 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:41.156963 9151 leader_election.cc:288] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:30:41.157121 9157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:41.158915 9157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [term 1 LEADER]: Becoming Leader. State: Replica: 3130f7d5587340e08f31403501075673, State: Running, Role: LEADER
I20260807 19:30:41.159754 9151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:30:41.159780 9157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } }
I20260807 19:30:41.164070 9158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3130f7d5587340e08f31403501075673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } } }
I20260807 19:30:41.164494 9159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3130f7d5587340e08f31403501075673. Latest consensus state: current_term: 1 leader_uuid: "3130f7d5587340e08f31403501075673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3130f7d5587340e08f31403501075673" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45547 } } }
I20260807 19:30:41.165213 9158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:41.165318 9159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:41.166602 9167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:30:41.174629 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:41.129619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.8.216.190:45547)
0807 19:30:41.130678 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:41.130694 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:41.132165 (+ 1471us) server_negotiation.cc:407] Connection header received
0807 19:30:41.142283 (+ 10118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:41.142305 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:41.142816 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:41.143333 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:41.146659 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.148071 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.149878 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.150610 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.150818 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:41.158733 (+ 7915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:41.158750 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:41.158773 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:41.158843 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:41.165744 (+ 6901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:41.167137 (+ 1393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:41.167147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:41.167156 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:41.167406 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:41.168187 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:41.168193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:41.168197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:41.169439 (+ 1242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:41.169953 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:41.171781 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0807 19:30:41.172456 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":515,"threads_started":1}
I20260807 19:30:41.179116 9167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:30:41.193624 9167 catalog_manager.cc:1347] Generated new cluster ID: 52051359e7c6400493c13a6c2cc2ddc6
I20260807 19:30:41.193775 9167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:30:41.225208 9167 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:30:41.226682 9167 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:30:41.235639 9167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3130f7d5587340e08f31403501075673: Generated new TSK 0
I20260807 19:30:41.236610 9167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:30:41.295467 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:32787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:41.696602 9180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:41.696946 9180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:41.697134 9180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:41.705272 9180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:41.705415 9180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:41.705549 9180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:30:41.716255 9180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:32787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:41.717644 9180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:41.719045 9180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:41.734774 9185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:41.735353 9188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:41.737035 9186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:41.742425 9180 server_base.cc:1034] running on GCE node
I20260807 19:30:41.743376 9180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:41.744637 9180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:41.746007 9180 hybrid_clock.cc:648] HybridClock initialized: now 1786131041745923 us; error 62 us; skew 500 ppm
I20260807 19:30:41.746454 9180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:41 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:41.751638 9180 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:30:41.754455 9180 webserver.cc:460] Webserver started at http://127.8.216.129:41917/ using document root <none> and password file <none>
I20260807 19:30:41.755265 9180 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:41.755429 9180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:41.755770 9180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:41.758919 9180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "2e1537fbfe18427aad83e4abcc7d9d01"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "e99c1066e409a6e92dbcfbfbe8a0f372"
server_key_iv: "9f79724b6973e83bafb79449c6c172d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.759721 9180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "2e1537fbfe18427aad83e4abcc7d9d01"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "e99c1066e409a6e92dbcfbfbe8a0f372"
server_key_iv: "9f79724b6973e83bafb79449c6c172d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.765764 9180 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260807 19:30:41.769603 9195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:41.771061 9180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:41.771237 9180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "2e1537fbfe18427aad83e4abcc7d9d01"
format_stamp: "Formatted at 2026-08-07 19:30:41 on dist-test-slave-8w9v"
server_key: "e99c1066e409a6e92dbcfbfbe8a0f372"
server_key_iv: "9f79724b6973e83bafb79449c6c172d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:41.771476 9180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:41.789811 9180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:41.794294 9180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:41.794627 9180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:41.795662 9180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:41.797314 9180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:41.797389 9180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:41.797585 9180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:41.797641 9180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:41.841032 9180 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:36137
I20260807 19:30:41.841050 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:36137 every 8 connection(s)
I20260807 19:30:41.842509 9180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:30:41.849231 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9180
I20260807 19:30:41.849500 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:30:41.850134 9058 external_mini_cluster.cc:1428] Setting key c3b63a4cce238cc30796d1d1c28ad958
Aug 07 19:30:41 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:41.866698 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:41.845501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44155 (local address 127.8.216.190:45547)
0807 19:30:41.846047 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:41.846052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:41.847036 (+ 984us) server_negotiation.cc:407] Connection header received
0807 19:30:41.849431 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:41.849439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:41.849524 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:41.849714 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:41.851468 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.852524 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.854260 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.854730 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.854874 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:41.858818 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:41.858828 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:41.858837 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:41.858882 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:41.862744 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:41.863537 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:41.863546 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:41.863552 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:41.863710 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:41.864295 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:41.864304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:41.864310 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:41.864728 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:41.864900 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:41.865848 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 19:30:41.866126 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":331,"threads_started":1}
I20260807 19:30:41.867483 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:41.846013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45547 (local address 127.8.216.129:44155)
0807 19:30:41.846865 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:41.846904 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:41.849055 (+ 2151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:41.849962 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:41.849970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:41.850572 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:41.851293 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:41.851312 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.852710 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.852714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:41.854108 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:41.854121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:41.854996 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:41.855005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:41.855277 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:41.856031 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:41.856055 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:41.858569 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:41.862910 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:41.862917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:41.862931 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:41.863405 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:41.863860 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:41.863863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:41.863866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:41.864191 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:41.864894 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:41.864905 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:41.865344 (+ 439us) client_negotiation.cc:769] Sending connection context
0807 19:30:41.865658 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 19:30:41.866083 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":387,"threads_started":1}
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:30:41.869237 9316 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45547
I20260807 19:30:41.869571 9316 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:41.870447 9316 heartbeater.cc:511] Master 127.8.216.190:45547 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
I20260807 19:30:41.872956 9113 ts_manager.cc:194] Registered new tserver with Master: 2e1537fbfe18427aad83e4abcc7d9d01 (127.8.216.129:36137)
I20260807 19:30:41.875288 9113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:44155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:30:41.896974 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:32787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:42.297421 9322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:42.297773 9322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:42.297958 9322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:42.305590 9322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:42.305742 9322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:42.306031 9322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:30:42.316372 9322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:32787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:42.317787 9322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:42.319271 9322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:42.333312 9327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:42.333235 9330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:42.333274 9328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:42.334843 9322 server_base.cc:1034] running on GCE node
I20260807 19:30:42.335515 9322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:42.336640 9322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:42.337882 9322 hybrid_clock.cc:648] HybridClock initialized: now 1786131042337832 us; error 44 us; skew 500 ppm
I20260807 19:30:42.338219 9322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:42 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:42.341820 9322 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:30:42.343572 9322 webserver.cc:460] Webserver started at http://127.8.216.130:39429/ using document root <none> and password file <none>
I20260807 19:30:42.344542 9322 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:42.344687 9322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:42.345052 9322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:42.348404 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "9aee4d8b0f2a49798214c327e8da2977"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "b041580bdc9ec7c7d8854356025816bc"
server_key_iv: "4261f85709b01ed07dccd3fa6a9a152f"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.349264 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "9aee4d8b0f2a49798214c327e8da2977"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "b041580bdc9ec7c7d8854356025816bc"
server_key_iv: "4261f85709b01ed07dccd3fa6a9a152f"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.355155 9322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:30:42.359434 9337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.361168 9322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:30:42.361404 9322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "9aee4d8b0f2a49798214c327e8da2977"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "b041580bdc9ec7c7d8854356025816bc"
server_key_iv: "4261f85709b01ed07dccd3fa6a9a152f"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.361665 9322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:42.386305 9322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:42.391110 9322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:42.391441 9322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:42.392575 9322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:42.394161 9322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:42.394271 9322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.394433 9322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:42.394508 9322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.436040 9322 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38481
I20260807 19:30:42.436106 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38481 every 8 connection(s)
I20260807 19:30:42.437449 9322 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:30:42.437916 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9322
I20260807 19:30:42.438153 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:30:42.438812 9058 external_mini_cluster.cc:1428] Setting key 9a6b7221f6b4ededf2af697c28723c96
Aug 07 19:30:42 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
I20260807 19:30:42.463277 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:42.440245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:59069 (local address 127.8.216.190:45547)
0807 19:30:42.440708 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:42.440714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:42.442081 (+ 1367us) server_negotiation.cc:407] Connection header received
0807 19:30:42.445250 (+ 3169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:42.445254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:42.445321 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:42.445424 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:42.447561 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:42.448876 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:42.450570 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:42.451405 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:42.451553 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:42.455419 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:42.455428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:42.455437 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:42.455483 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:42.459452 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:42.460551 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:42.460557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:42.460561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:42.460735 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:42.461315 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:42.461322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:42.461330 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:42.461677 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:42.461851 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:42.462665 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 19:30:42.462899 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":218,"threads_started":1}
I20260807 19:30:42.464445 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:42.440902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45547 (local address 127.8.216.130:59069)
0807 19:30:42.441898 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:42.441952 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:42.444905 (+ 2953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:42.445730 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:42.445745 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:42.446587 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:42.447353 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:42.447373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:42.449050 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:42.449053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:42.450415 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:42.450428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:42.451692 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:42.451697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:42.451920 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:42.452736 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:42.452761 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:42.455138 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:42.459654 (+ 4516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:42.459664 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:42.459687 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:42.460410 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:42.460863 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:42.460868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:42.460872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:42.461138 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:42.461810 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:42.461819 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:42.462253 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 19:30:42.462559 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 19:30:42.463153 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":408,"threads_started":1}
I20260807 19:30:42.465917 9463 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45547
I20260807 19:30:42.466238 9463 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:42.467132 9463 heartbeater.cc:511] Master 127.8.216.190:45547 requested a full tablet report, sending...
I20260807 19:30:42.468793 9113 ts_manager.cc:194] Registered new tserver with Master: 9aee4d8b0f2a49798214c327e8da2977 (127.8.216.130:38481)
I20260807 19:30:42.470124 9113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:59069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:30:42.487159 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:32787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:42.862280 9470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:42.862689 9470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:42.862880 9470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:42.870919 9470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:42.871097 9470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:42.871301 9470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:30:42.878489 9316 heartbeater.cc:503] Master 127.8.216.190:45547 was elected leader, sending a full tablet report...
I20260807 19:30:42.883733 9470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:32787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45547
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:42.885243 9470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:42.886670 9470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:42.902171 9475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:42.902197 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:42.902171 9478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:42.903030 9470 server_base.cc:1034] running on GCE node
I20260807 19:30:42.903716 9470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:42.904807 9470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:42.906039 9470 hybrid_clock.cc:648] HybridClock initialized: now 1786131042905966 us; error 43 us; skew 500 ppm
I20260807 19:30:42.906421 9470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:42 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:42.909785 9470 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:30:42.911280 9470 webserver.cc:460] Webserver started at http://127.8.216.131:43257/ using document root <none> and password file <none>
I20260807 19:30:42.912020 9470 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:42.912192 9470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:42.912551 9470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:42.915544 9470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "b6d663d131144ec98fc9da9f08c85b7c"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "e3e4831f70d1b59a363c6023aaf83e04"
server_key_iv: "af1f49a9d806b72b88727cb79fda3093"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.916422 9470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "b6d663d131144ec98fc9da9f08c85b7c"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "e3e4831f70d1b59a363c6023aaf83e04"
server_key_iv: "af1f49a9d806b72b88727cb79fda3093"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.921883 9470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:30:42.925506 9486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.926827 9470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:42.927019 9470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "b6d663d131144ec98fc9da9f08c85b7c"
format_stamp: "Formatted at 2026-08-07 19:30:42 on dist-test-slave-8w9v"
server_key: "e3e4831f70d1b59a363c6023aaf83e04"
server_key_iv: "af1f49a9d806b72b88727cb79fda3093"
server_key_version: "encryptionkey@0"
I20260807 19:30:42.927230 9470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:42.949659 9470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:42.953963 9470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:42.954293 9470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:42.955343 9470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:42.956923 9470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:42.957041 9470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.957197 9470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:42.957273 9470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:42.996608 9470 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:36571
I20260807 19:30:42.996714 9607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:36571 every 8 connection(s)
I20260807 19:30:42.998142 9470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:30:43.006775 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9470
I20260807 19:30:43.007016 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:30:43.007481 9058 external_mini_cluster.cc:1428] Setting key c9cea9355afb9fb01c164a0980d2142e
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:43.019740 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.000904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:56129 (local address 127.8.216.190:45547)
0807 19:30:43.001506 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.001511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.002586 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 19:30:43.004664 (+ 2078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.004670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.004769 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.004896 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.006561 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.007545 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.009919 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.010332 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.010485 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.014180 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.014185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.014188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.014222 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.016925 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.017650 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.017654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.017656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.017779 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.018184 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.018187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.018189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.018450 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.018567 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.019264 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.019441 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":330,"threads_started":1}
I20260807 19:30:43.020900 9611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.001525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45547 (local address 127.8.216.131:56129)
0807 19:30:43.002375 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:43.002423 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:43.004352 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:43.005158 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:43.005166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:43.005768 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:43.006362 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:43.006379 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.007768 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.007773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:43.009756 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:43.009774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.010563 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.010568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:43.010790 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.011559 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:43.011589 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:43.013972 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:43.017059 (+ 3087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:43.017066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:43.017080 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:43.017526 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:43.017883 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:43.017886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:43.017889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:43.018084 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:43.018555 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:43.018561 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:43.018902 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 19:30:43.019159 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 19:30:43.019580 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":310,"threads_started":1}
I20260807 19:30:43.022639 9608 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45547
I20260807 19:30:43.023006 9608 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:43.023926 9608 heartbeater.cc:511] Master 127.8.216.190:45547 requested a full tablet report, sending...
I20260807 19:30:43.025446 9113 ts_manager.cc:194] Registered new tserver with Master: b6d663d131144ec98fc9da9f08c85b7c (127.8.216.131:36571)
I20260807 19:30:43.026242 9113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:56129
I20260807 19:30:43.036926 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:43.085884 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.069168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.8.216.129:36137)
0807 19:30:43.070051 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.070058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.070087 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:30:43.070157 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.070164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.070385 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.070514 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.072085 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.073375 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.074650 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.075287 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.075404 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.079747 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.079759 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.079774 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.079817 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.082968 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.083675 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.083683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.083685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.083864 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.084355 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.084364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.084365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.084773 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.085053 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.085242 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.085487 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":568,"threads_started":1}
W20260807 19:30:43.087224 9223 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46636
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:43.139722 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.124221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.8.216.129:36137)
0807 19:30:43.124398 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.124402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.124738 (+ 336us) server_negotiation.cc:407] Connection header received
0807 19:30:43.125023 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.125028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.125136 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.125254 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.127180 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.127945 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.129455 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.129873 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.129983 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.134317 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.134322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.134325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.134362 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.137390 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.138042 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.138046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.138048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.138184 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.138516 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.138519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.138521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.138763 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.138942 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.139207 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.139385 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 19:30:43.141472 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9180
I20260807 19:30:43.154902 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.142547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.8.216.129:36137)
0807 19:30:43.142788 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.142793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.142992 (+ 199us) server_negotiation.cc:407] Connection header received
0807 19:30:43.143097 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.143101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.143156 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.143232 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.144769 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.145506 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.146918 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.147338 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.147433 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.149754 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.149759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.149763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.149795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.152621 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.153217 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.153221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.153223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.153354 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.153659 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.153662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.153664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.153911 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.154081 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.154331 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.154508 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:30:43.155767 9311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:43.240445 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9180
I20260807 19:30:43.259155 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9322
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:30:43.276986 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.260090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.8.216.130:38481)
0807 19:30:43.260781 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.260789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.260816 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:30:43.260878 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.260885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.261097 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.261229 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.262600 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.263610 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.265097 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.265740 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.265941 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.270457 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.270469 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.270485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.270534 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.274008 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.274699 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.274707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.274710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.274870 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.275268 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.275272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.275274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.275699 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.275993 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.276298 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.276526 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":325,"threads_started":1}
I20260807 19:30:43.278364 9458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:43.362062 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9322
I20260807 19:30:43.380764 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9470
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:30:43.398307 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:43.381920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.8.216.131:36571)
0807 19:30:43.382545 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:43.382552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:43.382581 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:30:43.382643 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:43.382650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:43.382861 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:43.383046 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:43.384608 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.385654 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.387006 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:43.387596 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:43.387749 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:43.392332 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:43.392347 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:43.392364 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:43.392409 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:43.395256 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.395987 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.395995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.395998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.396185 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:43.396584 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:43.396589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:43.396593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:43.396970 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:43.397296 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:43.397634 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:30:43.397929 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":316,"threads_started":1}
I20260807 19:30:43.399674 9602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:43.486102 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9470
I20260807 19:30:43.505160 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9079
I20260807 19:30:43.506428 9146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:43.611536 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9079
2026-08-07T19:30:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4335 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9630](info): set up 2 sockets
Aug 07 19:30:43 dist-test-slave-8w9v krb5kdc[9630](info): commencing operation
krb5kdc: starting...
W20260807 19:30:44.743300 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260807 19:30:44.864780 9058 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260807 19:30:45.035022 9058 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.170s user 0.000s sys 0.006s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260807 19:30:45.299439 9058 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.264s user 0.000s sys 0.005s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/test-user.keytab.
W20260807 19:30:46.012749 9058 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.713s user 0.000s sys 0.005s
Aug 07 19:30:46 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:30:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
W20260807 19:30:46.639635 9058 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.216.190: real 0.573s user 0.001s sys 0.009s
I20260807 19:30:46.640707 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:39829
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:46.979923 9643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:46.980366 9643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:46.980592 9643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:46.988391 9643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:30:46.988521 9643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:46.988667 9643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:46.988808 9643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:30:46.988885 9643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:30:46.999639 9643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:47.001225 9643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:47.002745 9643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:47.016631 9648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:47.016556 9651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:47.016675 9649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:47.018029 9643 server_base.cc:1034] running on GCE node
I20260807 19:30:47.019004 9643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:47.020783 9643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:47.022030 9643 hybrid_clock.cc:648] HybridClock initialized: now 1786131047021978 us; error 45 us; skew 500 ppm
I20260807 19:30:47.022487 9643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:47 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:47.033229 9643 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:30:47.035086 9643 webserver.cc:460] Webserver started at http://127.8.216.190:39643/ using document root <none> and password file <none>
I20260807 19:30:47.035844 9643 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:47.035960 9643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:47.036450 9643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:47.039842 9643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "9d1e072a3d134e5985b225310cae0358"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "5b06796faa9695c70a10fc010c0cc21e"
server_key_iv: "3fd444812f0613fcf3de2712f91bbbc6"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.040721 9643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "9d1e072a3d134e5985b225310cae0358"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "5b06796faa9695c70a10fc010c0cc21e"
server_key_iv: "3fd444812f0613fcf3de2712f91bbbc6"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.046298 9643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:30:47.049885 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:47.051208 9643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:30:47.051401 9643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "9d1e072a3d134e5985b225310cae0358"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "5b06796faa9695c70a10fc010c0cc21e"
server_key_iv: "3fd444812f0613fcf3de2712f91bbbc6"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.051621 9643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:47.082427 9643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:47.087195 9643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:47.087517 9643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:47.106683 9643 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:39829
I20260807 19:30:47.106706 9714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:39829 every 8 connection(s)
I20260807 19:30:47.108179 9643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:30:47.110489 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9643
I20260807 19:30:47.110718 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:30:47.111557 9058 external_mini_cluster.cc:1428] Setting key 712c534580bcbfed203ad62b2626e834
I20260807 19:30:47.112814 9715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:47.125074 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358: Bootstrap starting.
Aug 07 19:30:47 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:47.128372 9715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:47.129488 9715 log.cc:824] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:47.132623 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358: No bootstrap required, opened a new log
I20260807 19:30:47.135358 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:47.114000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.8.216.190:39829)
0807 19:30:47.114825 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:47.114837 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:47.114878 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:30:47.117147 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:47.117170 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:47.117604 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:47.118029 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:47.119562 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.120921 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.122230 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.122986 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.123220 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:47.127544 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:47.127553 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:47.127569 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:47.127609 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:47.130927 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:47.131659 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:47.131674 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:47.131682 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:47.131850 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:47.132326 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:47.132331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:47.132336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:47.132964 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:47.133312 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:47.133829 (+ 517us) server_negotiation.cc:299] Negotiation successful
0807 19:30:47.134285 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":461,"threads_started":1}
I20260807 19:30:47.141239 9715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } }
I20260807 19:30:47.141646 9715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:47.141759 9715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1e072a3d134e5985b225310cae0358, State: Initialized, Role: FOLLOWER
I20260807 19:30:47.142371 9715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } }
I20260807 19:30:47.142606 9715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:30:47.142760 9715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:30:47.142953 9715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:47.145670 9715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } }
I20260807 19:30:47.146174 9715 leader_election.cc:302] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d1e072a3d134e5985b225310cae0358; no voters:
I20260807 19:30:47.146761 9715 leader_election.cc:288] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:30:47.146925 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:47.147607 9721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [term 1 LEADER]: Becoming Leader. State: Replica: 9d1e072a3d134e5985b225310cae0358, State: Running, Role: LEADER
I20260807 19:30:47.148275 9721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } }
I20260807 19:30:47.149256 9715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:30:47.151142 9722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d1e072a3d134e5985b225310cae0358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } } }
I20260807 19:30:47.151600 9722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:47.151166 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d1e072a3d134e5985b225310cae0358. Latest consensus state: current_term: 1 leader_uuid: "9d1e072a3d134e5985b225310cae0358" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d1e072a3d134e5985b225310cae0358" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 39829 } } }
I20260807 19:30:47.151775 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:47.153288 9729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:30:47.158030 9729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:30:47.166786 9729 catalog_manager.cc:1347] Generated new cluster ID: 17c1558733d344ef90afea29d56ec15b
I20260807 19:30:47.167050 9729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:30:47.194061 9729 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:30:47.195477 9729 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:30:47.207365 9729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d1e072a3d134e5985b225310cae0358: Generated new TSK 0
I20260807 19:30:47.208346 9729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:30:47.264087 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:47.611400 9745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:47.611688 9745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:47.611860 9745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:47.619422 9745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:47.619575 9745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:47.619797 9745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:30:47.629901 9745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:47.631404 9745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:47.632859 9745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:47.646350 9753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:47.646464 9752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:47.646418 9755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:47.647641 9745 server_base.cc:1034] running on GCE node
I20260807 19:30:47.648351 9745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:47.649427 9745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:47.650640 9745 hybrid_clock.cc:648] HybridClock initialized: now 1786131047650579 us; error 56 us; skew 500 ppm
I20260807 19:30:47.650969 9745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:47 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:47.654390 9745 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:30:47.656108 9745 webserver.cc:460] Webserver started at http://127.8.216.129:36865/ using document root <none> and password file <none>
I20260807 19:30:47.656957 9745 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:47.657101 9745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:47.657420 9745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:47.660768 9745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "a429de6ee478462e9b6eb2a352b6b770"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "588b7d579c1810b325914e3230c5fca4"
server_key_iv: "3522dde5623290b59be5e2866622ac88"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.661602 9745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "a429de6ee478462e9b6eb2a352b6b770"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "588b7d579c1810b325914e3230c5fca4"
server_key_iv: "3522dde5623290b59be5e2866622ac88"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.666903 9745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:30:47.670446 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:47.671729 9745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:30:47.671914 9745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "a429de6ee478462e9b6eb2a352b6b770"
format_stamp: "Formatted at 2026-08-07 19:30:47 on dist-test-slave-8w9v"
server_key: "588b7d579c1810b325914e3230c5fca4"
server_key_iv: "3522dde5623290b59be5e2866622ac88"
server_key_version: "encryptionkey@0"
I20260807 19:30:47.672128 9745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:47.695082 9745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:47.699717 9745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:47.700038 9745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:47.701088 9745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:47.702476 9745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:47.702584 9745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:47.702740 9745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:47.702816 9745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:47.736635 9745 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:45677
I20260807 19:30:47.736689 9882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:45677 every 8 connection(s)
I20260807 19:30:47.737987 9745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:30:47.746322 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9745
I20260807 19:30:47.746544 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:30:47.747304 9058 external_mini_cluster.cc:1428] Setting key 72a1577db6323a990fbb64181aefd68e
Aug 07 19:30:47 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131047, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:47.762410 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:47.740562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49101 (local address 127.8.216.190:39829)
0807 19:30:47.741006 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:47.741010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:47.742023 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 19:30:47.743897 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:47.743903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:47.744007 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:47.744131 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:47.745654 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.746766 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.749530 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.750138 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.750258 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:47.754801 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:47.754808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:47.754812 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:47.754859 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:47.758489 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:47.759518 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:47.759527 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:47.759533 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:47.759692 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:47.760296 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:47.760304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:47.760311 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:47.760755 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:47.760917 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:47.761672 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 19:30:47.761926 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":205,"threads_started":1}
I20260807 19:30:47.763074 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:47.741132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:39829 (local address 127.8.216.129:49101)
0807 19:30:47.741853 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:47.741897 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:47.743606 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:47.744396 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:47.744404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:47.744915 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:47.745478 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:47.745494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.746984 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.746993 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:47.749321 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:47.749337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:47.750562 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:47.750569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:47.750842 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:47.751620 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:47.751661 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:47.754509 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:47.758643 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:47.758656 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:47.758677 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:47.759340 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:47.759816 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:47.759821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:47.759825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:47.760091 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:47.760892 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:47.760898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:47.761237 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 19:30:47.761486 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 19:30:47.761894 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":298,"threads_started":1}
I20260807 19:30:47.764910 9883 heartbeater.cc:347] Connected to a master server at 127.8.216.190:39829
I20260807 19:30:47.765277 9883 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:47.766108 9883 heartbeater.cc:511] Master 127.8.216.190:39829 requested a full tablet report, sending...
I20260807 19:30:47.768455 9676 ts_manager.cc:194] Registered new tserver with Master: a429de6ee478462e9b6eb2a352b6b770 (127.8.216.129:45677)
I20260807 19:30:47.770130 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:49101
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:30:47.800463 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:48.134168 9889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:48.134533 9889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:48.134716 9889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:48.142232 9889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:48.142416 9889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:48.142596 9889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:30:48.152906 9889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:48.154404 9889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:48.155830 9889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:48.170001 9895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:48.170040 9896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:48.170071 9899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:48.170992 9889 server_base.cc:1034] running on GCE node
I20260807 19:30:48.171636 9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:48.172696 9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:48.173900 9889 hybrid_clock.cc:648] HybridClock initialized: now 1786131048173855 us; error 32 us; skew 500 ppm
I20260807 19:30:48.174280 9889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:48 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131048, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:48.177726 9889 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:30:48.179338 9889 webserver.cc:460] Webserver started at http://127.8.216.130:41307/ using document root <none> and password file <none>
I20260807 19:30:48.180078 9889 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:48.180285 9889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:48.180629 9889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:48.183670 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "1ea01d346762186e85bba9b0e019abc8"
server_key_iv: "14a64e89be4128c037e5566214b159d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.184556 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "1ea01d346762186e85bba9b0e019abc8"
server_key_iv: "14a64e89be4128c037e5566214b159d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.190013 9889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 19:30:48.193601 9906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.194942 9889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:30:48.195134 9889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "1ea01d346762186e85bba9b0e019abc8"
server_key_iv: "14a64e89be4128c037e5566214b159d8"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.195353 9889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:48.226513 9889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:48.231299 9889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:48.231623 9889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:48.232734 9889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:48.234143 9889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:48.234256 9889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.234412 9889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:48.234498 9889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.269290 9889 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:41353
I20260807 19:30:48.269342 10026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:41353 every 8 connection(s)
I20260807 19:30:48.270761 9889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:30:48.280604 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 9889
I20260807 19:30:48.280794 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:30:48.281262 9058 external_mini_cluster.cc:1428] Setting key 348a371e4d483244af91839aca3381e2
Aug 07 19:30:48 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131048, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:48.292006 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.273338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:56637 (local address 127.8.216.190:39829)
0807 19:30:48.273787 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.273792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.274617 (+ 825us) server_negotiation.cc:407] Connection header received
0807 19:30:48.276672 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.276678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.276773 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.276900 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:48.278532 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.279250 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.281080 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.281580 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.281728 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.285519 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:48.285524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:48.285527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:48.285561 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:48.288379 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.289488 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.289496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.289507 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.289658 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.290152 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.290156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.290158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.290437 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:48.290599 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.291360 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.291600 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":243,"threads_started":1}
I20260807 19:30:48.293076 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.273739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:39829 (local address 127.8.216.130:56637)
0807 19:30:48.274440 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:48.274484 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:48.276377 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:48.277168 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:48.277177 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:48.277722 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:48.278328 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.278345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.279438 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.279448 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.280915 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.280930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.281825 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.281833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.282097 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.282835 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:48.282876 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:48.285254 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:48.288549 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:48.288559 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:48.288581 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:48.289325 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:48.289784 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:48.289790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:48.289794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:48.290011 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:48.290574 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:48.290582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:48.290958 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 19:30:48.291234 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:30:48.291834 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":287,"threads_started":1}
I20260807 19:30:48.294632 10027 heartbeater.cc:347] Connected to a master server at 127.8.216.190:39829
I20260807 19:30:48.295029 10027 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:48.295843 10027 heartbeater.cc:511] Master 127.8.216.190:39829 requested a full tablet report, sending...
I20260807 19:30:48.297521 9676 ts_manager.cc:194] Registered new tserver with Master: 1dfe4dd374ee4acfa2d5d21dfb62830e (127.8.216.130:41353)
I20260807 19:30:48.298382 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:56637
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:30:48.327138 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:48.662650 10033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:48.663004 10033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:48.663170 10033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:48.670639 10033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:48.670792 10033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:48.670984 10033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:30:48.681165 10033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:39829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:48.682642 10033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:48.684067 10033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:48.697621 10039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:48.697636 10040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:48.697736 10044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:48.698585 10033 server_base.cc:1034] running on GCE node
I20260807 19:30:48.699156 10033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:48.700289 10033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:48.701527 10033 hybrid_clock.cc:648] HybridClock initialized: now 1786131048701491 us; error 43 us; skew 500 ppm
I20260807 19:30:48.701812 10033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:48 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131048, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:48.704981 10033 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:30:48.706420 10033 webserver.cc:460] Webserver started at http://127.8.216.131:34841/ using document root <none> and password file <none>
I20260807 19:30:48.707147 10033 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:48.707281 10033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:48.707609 10033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:48.710595 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "8c0e0c59aa01483c848111b5b20e18f1"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "67a1e30e4785d5bb118a0b79919c410f"
server_key_iv: "510d7e872038aad8ebd115ffc3c568ba"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.711390 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "8c0e0c59aa01483c848111b5b20e18f1"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "67a1e30e4785d5bb118a0b79919c410f"
server_key_iv: "510d7e872038aad8ebd115ffc3c568ba"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.716639 10033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:30:48.720073 10051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.721467 10033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:48.721644 10033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "8c0e0c59aa01483c848111b5b20e18f1"
format_stamp: "Formatted at 2026-08-07 19:30:48 on dist-test-slave-8w9v"
server_key: "67a1e30e4785d5bb118a0b79919c410f"
server_key_iv: "510d7e872038aad8ebd115ffc3c568ba"
server_key_version: "encryptionkey@0"
I20260807 19:30:48.721843 10033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:48.744038 10033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:48.748889 10033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:48.749212 10033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:48.750344 10033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:30:48.751742 10033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:48.751818 10033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.751998 10033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:48.752050 10033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:48.773466 9883 heartbeater.cc:503] Master 127.8.216.190:39829 was elected leader, sending a full tablet report...
I20260807 19:30:48.786942 10033 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:33307
I20260807 19:30:48.786984 10173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:33307 every 8 connection(s)
I20260807 19:30:48.788470 10033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:30:48.797098 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10033
I20260807 19:30:48.797323 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:30:48.797945 9058 external_mini_cluster.cc:1428] Setting key 4d8bc9246dafff913ba02153bbb66b25
Aug 07 19:30:48 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131048, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:48.811970 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.791266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:43795 (local address 127.8.216.190:39829)
0807 19:30:48.791459 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.791464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.792881 (+ 1417us) server_negotiation.cc:407] Connection header received
0807 19:30:48.795123 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.795129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.795242 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.795366 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:48.797334 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.798499 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.800924 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.801604 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.801707 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.806074 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:48.806080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:48.806083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:48.806120 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:48.808993 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.809836 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.809840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.809842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.809996 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.810378 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.810381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.810382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.810660 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:48.810777 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.811448 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.811622 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:30:48.813114 10177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.791741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:39829 (local address 127.8.216.131:43795)
0807 19:30:48.792690 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:48.792739 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:48.794816 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:48.795614 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:48.795623 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:48.796329 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:48.797089 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.797110 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.798660 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.798664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.800773 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.800786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.801811 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.801815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.802027 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.802758 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:48.802782 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:48.805810 (+ 3028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:48.809135 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:48.809142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:48.809156 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:48.809717 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:48.810092 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:48.810095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:48.810097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:48.810277 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:48.810765 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:48.810771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:48.811108 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 19:30:48.811338 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 19:30:48.811759 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":451,"threads_started":1}
I20260807 19:30:48.814689 10174 heartbeater.cc:347] Connected to a master server at 127.8.216.190:39829
I20260807 19:30:48.815021 10174 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:48.815907 10174 heartbeater.cc:511] Master 127.8.216.190:39829 requested a full tablet report, sending...
I20260807 19:30:48.817498 9676 ts_manager.cc:194] Registered new tserver with Master: 8c0e0c59aa01483c848111b5b20e18f1 (127.8.216.131:33307)
I20260807 19:30:48.818444 9676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:43795
I20260807 19:30:48.826756 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:30:48.847594 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.834143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.8.216.190:39829)
0807 19:30:48.834311 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.834316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.834912 (+ 596us) server_negotiation.cc:407] Connection header received
0807 19:30:48.835145 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.835150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.835214 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.835322 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:48.837015 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.837787 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.839084 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.839539 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.839680 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.842040 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:48.842044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:48.842047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:48.842080 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:48.844849 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.845511 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.845518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.845522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.845664 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:48.846047 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:48.846052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:48.846055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:48.846371 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:48.846670 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.847040 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.847264 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:30:48.855010 9676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:30:48.862746 9676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:48.903136 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.877397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.8.216.130:41353)
0807 19:30:48.889561 (+ 12164us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.889573 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.892587 (+ 3014us) server_negotiation.cc:407] Connection header received
0807 19:30:48.892781 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.892794 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.893136 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.893379 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:48.894333 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.896485 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.899294 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.901439 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.901645 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.901886 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.902502 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.902717 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11946,"thread_start_us":11741,"threads_started":1}
I20260807 19:30:48.903815 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.876577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:41353 (local address 127.0.0.1:54552)
0807 19:30:48.891924 (+ 15347us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:48.891950 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:48.892152 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:48.893397 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:48.893404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:48.893436 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:48.894066 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.894081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.896624 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.896632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.898703 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.898759 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.901809 (+ 3050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.901818 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.902122 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.902236 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 19:30:48.902420 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 19:30:48.902655 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15161,"mutex_wait_us":1,"thread_start_us":13390,"threads_started":1}
I20260807 19:30:48.906435 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.876087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:45677 (local address 127.0.0.1:36322)
0807 19:30:48.890496 (+ 14409us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:48.890540 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:48.890801 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:48.894184 (+ 3383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:48.894188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:48.894213 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:48.895523 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.895537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.898258 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.898263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.903415 (+ 5152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.903428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.905506 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.905512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.905701 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.905734 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:30:48.905836 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:30:48.906003 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14109,"mutex_wait_us":341,"thread_start_us":17000,"threads_started":1}
I20260807 19:30:48.906461 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.876290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.8.216.129:45677)
0807 19:30:48.893064 (+ 16774us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.893074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.893106 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:30:48.893279 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.893289 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.893559 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.893837 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:48.895739 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.898100 (+ 2361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.903598 (+ 5498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.904921 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.905086 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.905349 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.905928 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.906106 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16551,"thread_start_us":16343,"threads_started":1}
I20260807 19:30:48.908234 9955 tablet_service.cc:1467] Processing CreateTablet for tablet 1bad13101fca456884c3308fd732301c (DEFAULT_TABLE table=test-table [id=4815117e59624e25bbd63209c8ce15ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:48.908880 10187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.876774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:33307 (local address 127.0.0.1:57984)
0807 19:30:48.893648 (+ 16874us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:48.893668 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:48.893926 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:48.895189 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:48.895194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:48.895216 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:48.895940 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.895954 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.898936 (+ 2982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.898962 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.904890 (+ 5928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:48.904904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.907983 (+ 3079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.907990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:48.908183 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.908213 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:30:48.908297 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:30:48.908430 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16679,"mutex_wait_us":1431,"thread_start_us":15483,"threads_started":1}
I20260807 19:30:48.909857 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bad13101fca456884c3308fd732301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:48.910115 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:48.877689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.8.216.131:33307)
0807 19:30:48.885751 (+ 8062us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:48.885764 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:48.893775 (+ 8011us) server_negotiation.cc:407] Connection header received
0807 19:30:48.894187 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:48.894195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:48.894483 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:48.894739 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:48.896270 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.898780 (+ 2510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.905098 (+ 6318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:48.907772 (+ 2674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:48.907937 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:48.909563 (+ 1626us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:48.909629 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 19:30:48.909756 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7755,"thread_start_us":7449,"threads_started":1}
I20260807 19:30:48.910305 9812 tablet_service.cc:1467] Processing CreateTablet for tablet 1bad13101fca456884c3308fd732301c (DEFAULT_TABLE table=test-table [id=4815117e59624e25bbd63209c8ce15ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:48.911835 9812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bad13101fca456884c3308fd732301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:48.912923 10103 tablet_service.cc:1467] Processing CreateTablet for tablet 1bad13101fca456884c3308fd732301c (DEFAULT_TABLE table=test-table [id=4815117e59624e25bbd63209c8ce15ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:48.913996 10103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bad13101fca456884c3308fd732301c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:48.926796 10194 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Bootstrap starting.
I20260807 19:30:48.928586 10196 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Bootstrap starting.
I20260807 19:30:48.931654 10196 tablet_bootstrap.cc:654] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:48.933175 10196 log.cc:824] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:48.933229 10195 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Bootstrap starting.
I20260807 19:30:48.933992 10194 tablet_bootstrap.cc:654] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:48.935041 10194 log.cc:824] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:48.938040 10195 tablet_bootstrap.cc:654] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:48.938377 10194 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: No bootstrap required, opened a new log
I20260807 19:30:48.938674 10194 ts_tablet_manager.cc:1397] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 19:30:48.940516 10196 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: No bootstrap required, opened a new log
I20260807 19:30:48.940928 10196 ts_tablet_manager.cc:1397] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260807 19:30:48.940869 10195 log.cc:824] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:48.954483 10196 raft_consensus.cc:348] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.955600 10196 raft_consensus.cc:374] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:48.955857 10196 raft_consensus.cc:729] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a429de6ee478462e9b6eb2a352b6b770, State: Initialized, Role: FOLLOWER
I20260807 19:30:48.955924 10195 tablet_bootstrap.cc:492] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: No bootstrap required, opened a new log
I20260807 19:30:48.956804 10195 ts_tablet_manager.cc:1397] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.008s
I20260807 19:30:48.957333 10196 consensus_queue.cc:260] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.961740 10196 ts_tablet_manager.cc:1428] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260807 19:30:48.962925 10194 raft_consensus.cc:348] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.963649 10194 raft_consensus.cc:374] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:48.963893 10194 raft_consensus.cc:729] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dfe4dd374ee4acfa2d5d21dfb62830e, State: Initialized, Role: FOLLOWER
I20260807 19:30:48.964859 10194 consensus_queue.cc:260] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.967178 10027 heartbeater.cc:503] Master 127.8.216.190:39829 was elected leader, sending a full tablet report...
I20260807 19:30:48.967718 10194 ts_tablet_manager.cc:1428] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Time spent starting tablet: real 0.029s user 0.013s sys 0.005s
I20260807 19:30:48.980926 10195 raft_consensus.cc:348] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.981798 10195 raft_consensus.cc:374] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:48.982048 10195 raft_consensus.cc:729] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0e0c59aa01483c848111b5b20e18f1, State: Initialized, Role: FOLLOWER
I20260807 19:30:48.983166 10195 consensus_queue.cc:260] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:48.985383 10174 heartbeater.cc:503] Master 127.8.216.190:39829 was elected leader, sending a full tablet report...
I20260807 19:30:48.986409 10195 ts_tablet_manager.cc:1428] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Time spent starting tablet: real 0.029s user 0.012s sys 0.013s
W20260807 19:30:48.990510 9884 tablet.cc:2369] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:30:49.022933 10028 tablet.cc:2369] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:30:49.040406 10176 tablet.cc:2369] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:30:49.051658 10201 raft_consensus.cc:482] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:49.051937 10201 raft_consensus.cc:504] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:49.053663 10201 leader_election.cc:288] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dfe4dd374ee4acfa2d5d21dfb62830e (127.8.216.130:41353), 8c0e0c59aa01483c848111b5b20e18f1 (127.8.216.131:33307)
I20260807 19:30:49.062968 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.054226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49299 (local address 127.8.216.131:33307)
0807 19:30:49.054509 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.054519 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.055805 (+ 1286us) server_negotiation.cc:407] Connection header received
0807 19:30:49.055960 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.055978 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.056084 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.056228 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:49.056999 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.058217 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.060626 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.061931 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.062142 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.062227 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.062661 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.062769 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:30:49.063227 10205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.053928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:33307 (local address 127.8.216.129:49299)
0807 19:30:49.055679 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:49.055696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:49.055823 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:49.056322 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:49.056326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:49.056348 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:49.056804 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:49.056819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.058345 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.058352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:49.060436 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:49.060448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.062192 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.062199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:49.062401 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.062519 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 19:30:49.062623 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:30:49.062780 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1640,"thread_start_us":1228,"threads_started":1}
I20260807 19:30:49.064888 10126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bad13101fca456884c3308fd732301c" candidate_uuid: "a429de6ee478462e9b6eb2a352b6b770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0e0c59aa01483c848111b5b20e18f1" is_pre_election: true
I20260807 19:30:49.065505 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.054365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:54921 (local address 127.8.216.130:41353)
0807 19:30:49.054672 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.054678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.056220 (+ 1542us) server_negotiation.cc:407] Connection header received
0807 19:30:49.056477 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.056485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.056597 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.056697 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:49.058002 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.059555 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.062531 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.064379 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.064596 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.064671 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.064949 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.065078 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 19:30:49.065934 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.053943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:41353 (local address 127.8.216.129:54921)
0807 19:30:49.056056 (+ 2113us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:49.056075 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:49.056260 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:49.056708 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:49.056715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:49.056734 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:49.057825 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:49.057840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.059770 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.059774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:49.062250 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:49.062262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.064598 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.064604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:49.064780 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.064804 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:30:49.064923 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:30:49.065056 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1884,"mutex_wait_us":601,"thread_start_us":695,"threads_started":1}
I20260807 19:30:49.065734 10126 raft_consensus.cc:2454] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a429de6ee478462e9b6eb2a352b6b770 in term 0.
I20260807 19:30:49.066977 9765 leader_election.cc:302] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c0e0c59aa01483c848111b5b20e18f1, a429de6ee478462e9b6eb2a352b6b770; no voters:
I20260807 19:30:49.067560 10201 raft_consensus.cc:2781] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:49.067773 10201 raft_consensus.cc:482] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:49.067469 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bad13101fca456884c3308fd732301c" candidate_uuid: "a429de6ee478462e9b6eb2a352b6b770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" is_pre_election: true
I20260807 19:30:49.067916 10201 raft_consensus.cc:3037] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:49.068424 9979 raft_consensus.cc:2454] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a429de6ee478462e9b6eb2a352b6b770 in term 0.
I20260807 19:30:49.070645 10201 raft_consensus.cc:504] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:49.071219 10201 leader_election.cc:288] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [CANDIDATE]: Term 1 election: Requested vote from peers 1dfe4dd374ee4acfa2d5d21dfb62830e (127.8.216.130:41353), 8c0e0c59aa01483c848111b5b20e18f1 (127.8.216.131:33307)
I20260807 19:30:49.071676 9979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bad13101fca456884c3308fd732301c" candidate_uuid: "a429de6ee478462e9b6eb2a352b6b770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e"
I20260807 19:30:49.071869 9979 raft_consensus.cc:3037] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:49.071975 10126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bad13101fca456884c3308fd732301c" candidate_uuid: "a429de6ee478462e9b6eb2a352b6b770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c0e0c59aa01483c848111b5b20e18f1"
I20260807 19:30:49.072305 10126 raft_consensus.cc:3037] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:49.074812 9979 raft_consensus.cc:2454] T 1bad13101fca456884c3308fd732301c P 1dfe4dd374ee4acfa2d5d21dfb62830e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a429de6ee478462e9b6eb2a352b6b770 in term 1.
I20260807 19:30:49.075380 9764 leader_election.cc:302] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dfe4dd374ee4acfa2d5d21dfb62830e, a429de6ee478462e9b6eb2a352b6b770; no voters:
I20260807 19:30:49.075878 10201 raft_consensus.cc:2781] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:49.076429 10126 raft_consensus.cc:2454] T 1bad13101fca456884c3308fd732301c P 8c0e0c59aa01483c848111b5b20e18f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a429de6ee478462e9b6eb2a352b6b770 in term 1.
I20260807 19:30:49.077744 10201 raft_consensus.cc:686] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [term 1 LEADER]: Becoming Leader. State: Replica: a429de6ee478462e9b6eb2a352b6b770, State: Running, Role: LEADER
I20260807 19:30:49.078519 10201 consensus_queue.cc:237] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } }
I20260807 19:30:49.085476 9676 catalog_manager.cc:5495] T 1bad13101fca456884c3308fd732301c P a429de6ee478462e9b6eb2a352b6b770 reported cstate change: term changed from 0 to 1, leader changed from <none> to a429de6ee478462e9b6eb2a352b6b770 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "a429de6ee478462e9b6eb2a352b6b770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a429de6ee478462e9b6eb2a352b6b770" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 45677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1dfe4dd374ee4acfa2d5d21dfb62830e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 41353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c0e0c59aa01483c848111b5b20e18f1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33307 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:49.162164 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.137459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36324 (local address 127.8.216.129:45677)
0807 19:30:49.137723 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.137728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.139141 (+ 1413us) server_negotiation.cc:407] Connection header received
0807 19:30:49.139406 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.139414 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.139496 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.139610 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.142887 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.144016 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.146055 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.146776 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.146944 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.153384 (+ 6440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.153402 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.153419 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.153470 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.158338 (+ 4868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.159239 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.159250 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.159256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.159650 (+ 394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.160184 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.160194 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.160198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.160730 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.161133 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.161337 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.161588 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:49.223373 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.201723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.8.216.129:45677)
0807 19:30:49.201998 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.202005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.202752 (+ 747us) server_negotiation.cc:407] Connection header received
0807 19:30:49.203022 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.203027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.203095 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.203205 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.205891 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.206933 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.208912 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.209641 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.209796 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.216008 (+ 6212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.216014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.216017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.216057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.219981 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.220764 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.220768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.220771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.220987 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.221554 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.221559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.221561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.221911 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.222328 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.222567 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.222833 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:49.285019 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.260536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.8.216.129:45677)
0807 19:30:49.260838 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.260848 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.261366 (+ 518us) server_negotiation.cc:407] Connection header received
0807 19:30:49.261514 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.261519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.261606 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.261699 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.264494 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.265368 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.267770 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.268469 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.268580 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.277074 (+ 8494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.277082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.277086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.277132 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.281505 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.282294 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.282301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.282304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.282534 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.283010 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.283016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.283019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.283470 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.283853 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.284227 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.284477 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 19:30:49.287303 9792 tablet_service.cc:2908] Rejecting scan request for tablet 1bad13101fca456884c3308fd732301c: Uninitialized: clean time has not yet been initialized
I20260807 19:30:49.291685 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9745
I20260807 19:30:49.314667 10189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.292756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.8.216.129:45677)
0807 19:30:49.292925 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.292930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.297021 (+ 4091us) server_negotiation.cc:407] Connection header received
0807 19:30:49.297225 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.297229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.297335 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.297435 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.299484 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.300773 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.302506 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.303348 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.303458 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.306994 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.306999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.307004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.307073 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.311146 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.311989 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.311993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.311995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.312265 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.312654 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.312657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.312659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.313085 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.313347 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.313719 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.314180 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:30:49.315747 9878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:49.411415 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9745
I20260807 19:30:49.431228 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9889
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:30:49.447116 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.431973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.8.216.130:41353)
0807 19:30:49.432185 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.432194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.432220 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:30:49.432471 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.432475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.432540 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.432657 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.434844 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.435632 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.437076 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.437480 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.437581 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.441642 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.441655 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.441670 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.441701 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.444647 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.445261 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.445268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.445270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.445388 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.445739 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.445742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.445744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.446108 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.446319 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.446589 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.446752 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:30:49.447965 10022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:49.540979 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9889
I20260807 19:30:49.559974 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10033
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[9630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:30:49.574707 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:49.560877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.8.216.131:33307)
0807 19:30:49.561071 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:49.561076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:49.561097 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:30:49.561195 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:49.561198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:49.561261 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:49.561344 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:49.562627 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.563439 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.564774 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:49.565192 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:49.565290 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:49.569545 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:49.569560 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:49.569577 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:49.569616 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:49.572272 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.572857 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.572863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.572866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.573006 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:49.573331 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:49.573333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:49.573335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:49.573677 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:49.573882 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:49.574122 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 19:30:49.574294 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:30:49.575354 10169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:49.664289 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10033
I20260807 19:30:49.683293 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 9643
I20260807 19:30:49.684618 9709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:49.797075 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 9643
2026-08-07T19:30:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (6186 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[10229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[10229](info): set up 2 sockets
Aug 07 19:30:49 dist-test-slave-8w9v krb5kdc[10229](info): commencing operation
krb5kdc: starting...
W20260807 19:30:50.928529 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:30:51 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:30:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:30:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:30:51.133915 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46575
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:46575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:51.551753 10242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:51.552137 10242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:51.552346 10242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:51.559659 10242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:30:51.559816 10242 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:30:51.559928 10242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:51.560045 10242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:51.560163 10242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:30:51.560258 10242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:30:51.570359 10242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:46575
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:51.571663 10242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:51.573068 10242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:51.585712 10250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:51.585774 10252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:51.585924 10248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:51.586212 10242 server_base.cc:1034] running on GCE node
I20260807 19:30:51.587293 10242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:51.588817 10242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:51.590076 10242 hybrid_clock.cc:648] HybridClock initialized: now 1786131051589999 us; error 53 us; skew 500 ppm
I20260807 19:30:51.590418 10242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:51 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131051, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:51.593622 10242 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:30:51.595109 10242 webserver.cc:460] Webserver started at http://127.8.216.190:41847/ using document root <none> and password file <none>
I20260807 19:30:51.595821 10242 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:51.595954 10242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:51.596364 10242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:51.599308 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "1e15ef756590426e86d4c7aedb8b8014"
format_stamp: "Formatted at 2026-08-07 19:30:51 on dist-test-slave-8w9v"
server_key: "78fdb0bd920bd2644e3cbc6ae6598b6a"
server_key_iv: "334a19ca5d9536e849de6c3bd0d1d517"
server_key_version: "encryptionkey@0"
I20260807 19:30:51.600175 10242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "1e15ef756590426e86d4c7aedb8b8014"
format_stamp: "Formatted at 2026-08-07 19:30:51 on dist-test-slave-8w9v"
server_key: "78fdb0bd920bd2644e3cbc6ae6598b6a"
server_key_iv: "334a19ca5d9536e849de6c3bd0d1d517"
server_key_version: "encryptionkey@0"
I20260807 19:30:51.605386 10242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:30:51.608865 10259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:51.610178 10242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:51.610388 10242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "1e15ef756590426e86d4c7aedb8b8014"
format_stamp: "Formatted at 2026-08-07 19:30:51 on dist-test-slave-8w9v"
server_key: "78fdb0bd920bd2644e3cbc6ae6598b6a"
server_key_iv: "334a19ca5d9536e849de6c3bd0d1d517"
server_key_version: "encryptionkey@0"
I20260807 19:30:51.610607 10242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:51.631155 10242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:51.635586 10242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:51.635910 10242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:51.659737 10242 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:46575
I20260807 19:30:51.659749 10323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:46575 every 8 connection(s)
I20260807 19:30:51.661324 10242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:30:51.665431 10324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:51.665639 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10242
I20260807 19:30:51.665874 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:30:51.666610 9058 external_mini_cluster.cc:1428] Setting key 52d79a97b821f84e64169640cc73a140
I20260807 19:30:51.679569 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Bootstrap starting.
Aug 07 19:30:51 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:51.683573 10324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:51.685057 10324 log.cc:824] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:51.688701 10324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: No bootstrap required, opened a new log
I20260807 19:30:51.692677 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:51.668779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.8.216.190:46575)
0807 19:30:51.669729 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:51.669741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:51.669788 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:30:51.671581 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:51.671606 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:51.672049 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:51.672507 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:51.674381 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:51.675780 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:51.677631 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:51.678387 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:51.678543 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:51.683706 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:51.683718 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:51.683740 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:51.683791 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:51.687905 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:51.688744 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:51.688754 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:51.688763 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:51.689003 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:51.689391 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:51.689396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:51.689399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:51.690131 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:51.690455 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:51.690763 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 19:30:51.691232 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":464,"threads_started":1}
I20260807 19:30:51.697556 10324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } }
I20260807 19:30:51.697897 10324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:51.698011 10324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e15ef756590426e86d4c7aedb8b8014, State: Initialized, Role: FOLLOWER
I20260807 19:30:51.698676 10324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } }
I20260807 19:30:51.698947 10324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:30:51.699136 10324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:30:51.699303 10324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:51.701637 10324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } }
I20260807 19:30:51.702163 10324 leader_election.cc:302] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e15ef756590426e86d4c7aedb8b8014; no voters:
I20260807 19:30:51.702790 10324 leader_election.cc:288] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:30:51.702965 10329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:51.704466 10329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [term 1 LEADER]: Becoming Leader. State: Replica: 1e15ef756590426e86d4c7aedb8b8014, State: Running, Role: LEADER
I20260807 19:30:51.705170 10324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:30:51.705060 10329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } }
I20260807 19:30:51.708057 10331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e15ef756590426e86d4c7aedb8b8014. Latest consensus state: current_term: 1 leader_uuid: "1e15ef756590426e86d4c7aedb8b8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } } }
I20260807 19:30:51.708536 10331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:51.708212 10330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e15ef756590426e86d4c7aedb8b8014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e15ef756590426e86d4c7aedb8b8014" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46575 } } }
I20260807 19:30:51.708794 10330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:51.711041 10338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:30:51.716778 10338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:30:51.725910 10338 catalog_manager.cc:1347] Generated new cluster ID: c991a24fdd8c4ef093d5efe09e2cf1f4
I20260807 19:30:51.726104 10338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:30:51.738996 10338 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:30:51.741034 10338 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:30:51.751746 10338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Generated new TSK 0
I20260807 19:30:51.753119 10338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:30:51.807803 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:46575
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:52.147324 10352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:52.147683 10352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:52.147854 10352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:52.155246 10352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:52.155412 10352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:52.155555 10352 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:52.155701 10352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:30:52.166143 10352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:52.167747 10352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:52.169255 10352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:52.183790 10359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:52.183851 10358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:52.183827 10361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:52.184795 10352 server_base.cc:1034] running on GCE node
I20260807 19:30:52.185494 10352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:52.186414 10352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:52.187692 10352 hybrid_clock.cc:648] HybridClock initialized: now 1786131052187612 us; error 35 us; skew 500 ppm
I20260807 19:30:52.188016 10352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:52 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131052, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:52.195344 10352 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:30:52.196893 10352 webserver.cc:460] Webserver started at http://127.8.216.129:45857/ using document root <none> and password file <none>
I20260807 19:30:52.197584 10352 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:52.197674 10352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:52.198020 10352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:52.201053 10352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "ea340443b6ab4c93b68077e48fce246e"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "fff8d3b039e40b230dd9c567ac180ea9"
server_key_iv: "1bf5acf0706201c86416f26055db6e98"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.201812 10352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "ea340443b6ab4c93b68077e48fce246e"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "fff8d3b039e40b230dd9c567ac180ea9"
server_key_iv: "1bf5acf0706201c86416f26055db6e98"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.207195 10352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:30:52.210887 10369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:52.212431 10352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:30:52.212615 10352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "ea340443b6ab4c93b68077e48fce246e"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "fff8d3b039e40b230dd9c567ac180ea9"
server_key_iv: "1bf5acf0706201c86416f26055db6e98"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.212852 10352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:52.231732 10352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:52.236742 10352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:52.237056 10352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:52.240024 10352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:52.240170 10352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:52.240335 10352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:52.240406 10352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:52 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131052, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:52.265743 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.240501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:40621 (local address 127.8.216.190:46575)
0807 19:30:52.241424 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:52.241428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:52.242199 (+ 771us) server_negotiation.cc:407] Connection header received
0807 19:30:52.244350 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:52.244356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:52.244455 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:52.244598 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:52.246925 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.248072 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.251064 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.251707 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.251881 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.258240 (+ 6359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:52.258250 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:52.258257 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:52.258316 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:52.261866 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.262715 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.262721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.262725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.262868 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.263372 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.263377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.263381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.263688 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:52.263854 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:52.264827 (+ 973us) server_negotiation.cc:299] Negotiation successful
0807 19:30:52.265183 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":693,"threads_started":1}
I20260807 19:30:52.266554 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.241203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.129:40621)
0807 19:30:52.242031 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:52.242069 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:52.244005 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:52.244870 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:52.244879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:52.245667 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:52.246703 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.246733 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.248273 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.248282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.250779 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.250797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.252274 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.252281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.252538 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.253558 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:52.253594 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:52.257883 (+ 4289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:52.262055 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.262066 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.262087 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.262572 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.263002 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.263007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.263011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.263240 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.263827 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:52.263836 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:52.264334 (+ 498us) client_negotiation.cc:769] Sending connection context
0807 19:30:52.264670 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 19:30:52.265187 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":359,"threads_started":1}
I20260807 19:30:52.289805 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.273350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.129:46487)
0807 19:30:52.274085 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:52.274102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:52.274226 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:52.274648 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:52.274651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:52.275120 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:52.275416 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.275425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.277134 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.277140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.278686 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.278700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.279957 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.279962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.280222 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.280951 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:52.280969 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:52.282541 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:52.286340 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.286347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.286357 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.287007 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.287492 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.287498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.287501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.287635 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.288200 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:52.288205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:52.288431 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 19:30:52.288618 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 19:30:52.289007 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":395,"threads_started":1}
I20260807 19:30:52.290035 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.273402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:46487 (local address 127.8.216.190:46575)
0807 19:30:52.273652 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:52.273662 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:52.274192 (+ 530us) server_negotiation.cc:407] Connection header received
0807 19:30:52.274354 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:52.274360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:52.274475 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:52.274611 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:52.275567 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.276942 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.278927 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.279523 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.279675 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.282824 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:52.282829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:52.282832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:52.282868 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:52.286134 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.287196 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.287201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.287205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.287362 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.287749 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.287756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.287759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.288046 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:52.288248 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:52.289512 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0807 19:30:52.289741 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:30:52.296713 10352 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:32791
I20260807 19:30:52.296756 10499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:32791 every 8 connection(s)
I20260807 19:30:52.297931 10352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:30:52.300117 10501 heartbeater.cc:347] Connected to a master server at 127.8.216.190:46575
I20260807 19:30:52.300326 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10352
I20260807 19:30:52.300426 10501 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:52.300544 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:30:52.301116 10501 heartbeater.cc:511] Master 127.8.216.190:46575 requested a full tablet report, sending...
I20260807 19:30:52.301213 9058 external_mini_cluster.cc:1428] Setting key d5d2f99a13ce210927f3ef4d86322483
I20260807 19:30:52.303184 10276 ts_manager.cc:194] Registered new tserver with Master: ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791)
I20260807 19:30:52.304746 10276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:40621
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:30:52.347826 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:46575
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:52.689141 10506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:52.689429 10506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:52.689621 10506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:52.697021 10506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:52.697135 10506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:52.697315 10506 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:52.697453 10506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:30:52.707832 10506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:52.709188 10506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:52.710678 10506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:52.723915 10512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:52.724004 10513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:52.724004 10515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:52.725133 10506 server_base.cc:1034] running on GCE node
I20260807 19:30:52.725780 10506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:52.726742 10506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:52.727996 10506 hybrid_clock.cc:648] HybridClock initialized: now 1786131052727934 us; error 38 us; skew 500 ppm
I20260807 19:30:52.728345 10506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:52 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131052, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:52.731564 10506 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:30:52.733078 10506 webserver.cc:460] Webserver started at http://127.8.216.130:37347/ using document root <none> and password file <none>
I20260807 19:30:52.733784 10506 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:52.733925 10506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:52.734256 10506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:52.737346 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "b1bf7c2a3ade47a2b7837491a1a5276d"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "691f87770a1677b62df0dc2fd5335fed"
server_key_iv: "69fc0b2ebe562805302d69102b157cce"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.738159 10506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "b1bf7c2a3ade47a2b7837491a1a5276d"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "691f87770a1677b62df0dc2fd5335fed"
server_key_iv: "69fc0b2ebe562805302d69102b157cce"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.743235 10506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:30:52.746794 10522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:52.748114 10506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:30:52.748315 10506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "b1bf7c2a3ade47a2b7837491a1a5276d"
format_stamp: "Formatted at 2026-08-07 19:30:52 on dist-test-slave-8w9v"
server_key: "691f87770a1677b62df0dc2fd5335fed"
server_key_iv: "69fc0b2ebe562805302d69102b157cce"
server_key_version: "encryptionkey@0"
I20260807 19:30:52.748550 10506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:52.763039 10506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:52.768189 10506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:52.768550 10506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:52.771755 10506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:52.771874 10506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:52.772044 10506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:52.772148 10506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:52 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131052, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:52.797958 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.772405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:44627 (local address 127.8.216.190:46575)
0807 19:30:52.772626 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:52.772633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:52.774165 (+ 1532us) server_negotiation.cc:407] Connection header received
0807 19:30:52.776796 (+ 2631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:52.776802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:52.776907 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:52.777020 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:52.779391 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.780536 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.783219 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.783863 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.784012 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.788623 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:52.788629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:52.788632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:52.788667 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:52.793432 (+ 4765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.794713 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.794717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.794719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.794919 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.795535 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.795539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.795541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.795828 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:52.795955 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:52.797217 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0807 19:30:52.797500 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 19:30:52.800220 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.773021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.130:44627)
0807 19:30:52.773955 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:52.774006 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:52.776374 (+ 2368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:52.777258 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:52.777266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:52.778181 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:52.779132 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.779168 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.780712 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.780717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.783054 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.783067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.784171 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.784181 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.784502 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.785383 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:52.785413 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:52.788369 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:52.793649 (+ 5280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.793665 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.793692 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.794558 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.795053 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.795062 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.795067 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.795394 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.795980 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:52.795991 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:52.796666 (+ 675us) client_negotiation.cc:769] Sending connection context
0807 19:30:52.797127 (+ 461us) client_negotiation.cc:240] Negotiation successful
0807 19:30:52.797909 (+ 782us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":420,"threads_started":1}
I20260807 19:30:52.824806 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.807037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:43263 (local address 127.8.216.190:46575)
0807 19:30:52.807253 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:52.807261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:52.808802 (+ 1541us) server_negotiation.cc:407] Connection header received
0807 19:30:52.809035 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:52.809042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:52.809144 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:52.809253 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:52.810874 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.811781 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.814484 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.814948 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.815051 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.818125 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:52.818130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:52.818133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:52.818169 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:52.821456 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.822567 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.822571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.822574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.822710 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:52.823177 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:52.823180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:52.823182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:52.823528 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:52.823645 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:52.824318 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 19:30:52.824563 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:30:52.825328 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:52.807098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.130:43263)
0807 19:30:52.808668 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:52.808688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:52.808872 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:52.809334 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:52.809343 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:52.810144 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:52.810670 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.810688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.811939 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.811947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.814296 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:52.814319 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:52.815158 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:52.815168 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:52.815406 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:52.816369 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:52.816400 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:52.817896 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:52.821613 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.821621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.821625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.822399 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.822831 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:52.822839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:52.822845 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:52.823052 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:52.823650 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:52.823658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:52.823936 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 19:30:52.824161 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 19:30:52.824629 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"thread_start_us":1249,"threads_started":1}
I20260807 19:30:52.834939 10506 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:43251
I20260807 19:30:52.834982 10654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:43251 every 8 connection(s)
I20260807 19:30:52.836325 10506 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:30:52.838196 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10506
I20260807 19:30:52.838379 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:30:52.838653 10655 heartbeater.cc:347] Connected to a master server at 127.8.216.190:46575
I20260807 19:30:52.838888 10655 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:52.838954 9058 external_mini_cluster.cc:1428] Setting key 4335ad5d203c5d9c07daf605ff1975c7
I20260807 19:30:52.839557 10655 heartbeater.cc:511] Master 127.8.216.190:46575 requested a full tablet report, sending...
I20260807 19:30:52.841071 10277 ts_manager.cc:194] Registered new tserver with Master: b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130:43251)
I20260807 19:30:52.841940 10277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:44627
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:30:52.884557 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:46575
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:53.219331 10660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:53.219630 10660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:53.219798 10660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:53.227154 10660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:53.227284 10660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:53.227370 10660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:53.227527 10660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:30:53.237653 10660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:46575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:53.239079 10660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:53.240520 10660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:53.254104 10665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:53.254053 10666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:53.254011 10668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:53.255081 10660 server_base.cc:1034] running on GCE node
I20260807 19:30:53.255678 10660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:53.256735 10660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:53.257951 10660 hybrid_clock.cc:648] HybridClock initialized: now 1786131053257867 us; error 57 us; skew 500 ppm
I20260807 19:30:53.258260 10660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:53 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:53.261613 10660 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:30:53.263096 10660 webserver.cc:460] Webserver started at http://127.8.216.131:42147/ using document root <none> and password file <none>
I20260807 19:30:53.263832 10660 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:53.263957 10660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:53.264382 10660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:53.267341 10660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "86d076d6991540a494c25da074525e01"
format_stamp: "Formatted at 2026-08-07 19:30:53 on dist-test-slave-8w9v"
server_key: "114ab4bfa936967acf2a4fc668359e09"
server_key_iv: "38c481b779a9ae5c98a35a8d026fdb94"
server_key_version: "encryptionkey@0"
I20260807 19:30:53.268234 10660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "86d076d6991540a494c25da074525e01"
format_stamp: "Formatted at 2026-08-07 19:30:53 on dist-test-slave-8w9v"
server_key: "114ab4bfa936967acf2a4fc668359e09"
server_key_iv: "38c481b779a9ae5c98a35a8d026fdb94"
server_key_version: "encryptionkey@0"
I20260807 19:30:53.273671 10660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:30:53.277068 10675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:53.278429 10660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:30:53.278568 10660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "86d076d6991540a494c25da074525e01"
format_stamp: "Formatted at 2026-08-07 19:30:53 on dist-test-slave-8w9v"
server_key: "114ab4bfa936967acf2a4fc668359e09"
server_key_iv: "38c481b779a9ae5c98a35a8d026fdb94"
server_key_version: "encryptionkey@0"
I20260807 19:30:53.278750 10660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:53.302572 10660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:53.307284 10660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:53.307593 10660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:53.308223 10501 heartbeater.cc:503] Master 127.8.216.190:46575 was elected leader, sending a full tablet report...
I20260807 19:30:53.311228 10660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:53.311340 10660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:53.311486 10660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:53.311580 10660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:53 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:53.338673 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.311242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:44749 (local address 127.8.216.190:46575)
0807 19:30:53.311423 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.311427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.313046 (+ 1619us) server_negotiation.cc:407] Connection header received
0807 19:30:53.315438 (+ 2392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.315443 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.315541 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.315660 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:53.318090 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.319472 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.321731 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.322508 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.322667 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.328541 (+ 5874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:53.328551 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:53.328560 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:53.328620 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:53.333188 (+ 4568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.334971 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.334981 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.334984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.335125 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.335726 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.335732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.335735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.336271 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:53.336447 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.337867 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.338183 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:30:53.340651 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.312071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.131:44749)
0807 19:30:53.312901 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.312940 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.315069 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.315995 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.316006 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.316962 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:53.317890 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.317911 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.319658 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.319662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.321549 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.321568 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.322796 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.322807 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.323153 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.323944 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:53.323994 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:53.328146 (+ 4152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:53.333466 (+ 5320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.333480 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.333507 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.334782 (+ 1275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.335342 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.335347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.335351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.335579 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.336553 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:53.336566 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:53.337280 (+ 714us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.337709 (+ 429us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.338501 (+ 792us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":344,"threads_started":1}
I20260807 19:30:53.357802 10660 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:39161
I20260807 19:30:53.357928 10799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:39161 every 8 connection(s)
I20260807 19:30:53.359333 10660 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:30:53.361735 10802 heartbeater.cc:347] Connected to a master server at 127.8.216.190:46575
I20260807 19:30:53.361979 10802 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:53.362670 10802 heartbeater.cc:511] Master 127.8.216.190:46575 requested a full tablet report, sending...
I20260807 19:30:53.364192 10277 ts_manager.cc:194] Registered new tserver with Master: 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:53.364974 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10660
I20260807 19:30:53.365176 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:30:53.365363 10277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:44749
I20260807 19:30:53.365736 9058 external_mini_cluster.cc:1428] Setting key 3b609e95831cbc50e50065ec421fb423
I20260807 19:30:53.368942 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:30:53.378918 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.359077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:55443 (local address 127.8.216.190:46575)
0807 19:30:53.359342 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.359352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.364773 (+ 5421us) server_negotiation.cc:407] Connection header received
0807 19:30:53.364955 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.364961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.365037 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.365145 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:53.366414 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.367434 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.369610 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.370113 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.370210 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.372557 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:53.372566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:53.372573 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:53.372614 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:53.376034 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.376916 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.376921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.376925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.377066 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.377445 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.377450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.377453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.377741 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:53.377909 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.378316 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.378547 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":518,"server-negotiator.queue_time_us":80}
I20260807 19:30:53.378979 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.358960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.8.216.131:55443)
0807 19:30:53.364644 (+ 5684us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.364667 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.364825 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.365287 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.365291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.365878 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:53.366256 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.366269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.367805 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.367810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.369478 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.369493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.370296 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.370302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.370486 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.371165 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:53.371188 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:53.372340 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:53.376238 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.376244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.376250 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.376793 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.377183 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.377188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.377191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.377324 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.377875 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:53.377879 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:53.378059 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.378202 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.378516 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5500,"thread_start_us":5350,"threads_started":1}
Aug 07 19:30:53 dist-test-slave-8w9v krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:30:53 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:53.420667 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.406497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.8.216.190:46575)
0807 19:30:53.406746 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.406755 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.407086 (+ 331us) server_negotiation.cc:407] Connection header received
0807 19:30:53.407318 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.407323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.407396 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.407504 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:53.409004 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.409753 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.411118 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.411548 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.411687 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.415590 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:53.415595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:53.415598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:53.415630 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:53.418414 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.418997 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.419000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.419003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.419126 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.419455 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.419458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.419460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.419723 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:53.419915 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.420221 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.420389 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:30:53.424762 10277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:30:53.431756 10277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:53.459632 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.444801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.8.216.129:32791)
0807 19:30:53.449273 (+ 4472us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.449284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.450066 (+ 782us) server_negotiation.cc:407] Connection header received
0807 19:30:53.450495 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.450504 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.450765 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.450933 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.452001 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.453639 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.456509 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.458046 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.458174 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.458424 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.459118 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.459317 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4306,"thread_start_us":4169,"threads_started":1}
I20260807 19:30:53.460925 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.444597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:32791 (local address 127.0.0.1:32844)
0807 19:30:53.449884 (+ 5287us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.449955 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.450246 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.451248 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.451255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.451293 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.451767 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.451781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.453819 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.453824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.456318 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.456331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.458623 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.458630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.458835 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.458887 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.460264 (+ 1377us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.460489 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4954,"mutex_wait_us":2,"spinlock_wait_cycles":23168,"thread_start_us":4741,"threads_started":1}
I20260807 19:30:53.462229 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.444804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43251 (local address 127.0.0.1:47388)
0807 19:30:53.449884 (+ 5080us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.449950 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.450246 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.451248 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.451255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.451295 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.453016 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.453031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.454869 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.454873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.458188 (+ 3315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.458203 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.460352 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.460358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.460543 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.460576 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.461865 (+ 1289us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.461972 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4897,"mutex_wait_us":1199,"spinlock_wait_cycles":12288,"thread_start_us":4749,"threads_started":1}
I20260807 19:30:53.462446 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.445422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.8.216.130:43251)
0807 19:30:53.446081 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.446094 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.450067 (+ 3973us) server_negotiation.cc:407] Connection header received
0807 19:30:53.450496 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.450504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.450765 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.450933 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.453203 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.454650 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.458372 (+ 3722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.459810 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.459915 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.460114 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.461955 (+ 1841us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.462150 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":284,"threads_started":1}
I20260807 19:30:53.465679 10425 tablet_service.cc:1467] Processing CreateTablet for tablet ebdfe090cd544cf3900d85679cce7bcf (DEFAULT_TABLE table=test-table [id=33a63b65a6074527bf2cba44d4a459b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.465983 10578 tablet_service.cc:1467] Processing CreateTablet for tablet ebdfe090cd544cf3900d85679cce7bcf (DEFAULT_TABLE table=test-table [id=33a63b65a6074527bf2cba44d4a459b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.467314 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebdfe090cd544cf3900d85679cce7bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.467914 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebdfe090cd544cf3900d85679cce7bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.472342 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.450268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39161 (local address 127.0.0.1:40368)
0807 19:30:53.461377 (+ 11109us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.461397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.461501 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.465338 (+ 3837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.465344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.465374 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.465553 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.465564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.467576 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.467581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.469554 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.469571 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.471450 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.471456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.471654 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.471690 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.471807 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.471984 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11012,"thread_start_us":11176,"threads_started":1}
I20260807 19:30:53.472436 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.450681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40368 (local address 127.8.216.131:39161)
0807 19:30:53.464706 (+ 14025us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.464714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.464749 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:30:53.464843 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.464850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.465113 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.465316 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.465755 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.467417 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.469753 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.471260 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.471400 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.471637 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.471867 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.472063 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13851,"thread_start_us":13661,"threads_started":1}
I20260807 19:30:53.476893 10727 tablet_service.cc:1467] Processing CreateTablet for tablet ebdfe090cd544cf3900d85679cce7bcf (DEFAULT_TABLE table=test-table [id=33a63b65a6074527bf2cba44d4a459b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.478487 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebdfe090cd544cf3900d85679cce7bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.480427 10821 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Bootstrap starting.
I20260807 19:30:53.484231 10821 tablet_bootstrap.cc:654] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.485422 10821 log.cc:824] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:53.488291 10822 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Bootstrap starting.
I20260807 19:30:53.488449 10821 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: No bootstrap required, opened a new log
I20260807 19:30:53.488703 10821 ts_tablet_manager.cc:1397] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260807 19:30:53.492077 10822 tablet_bootstrap.cc:654] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.493474 10822 log.cc:824] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:53.496694 10821 raft_consensus.cc:348] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.497089 10821 raft_consensus.cc:374] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.497200 10821 raft_consensus.cc:729] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bf7c2a3ade47a2b7837491a1a5276d, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.497879 10825 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Bootstrap starting.
I20260807 19:30:53.498497 10822 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: No bootstrap required, opened a new log
I20260807 19:30:53.497990 10821 consensus_queue.cc:260] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.498831 10822 ts_tablet_manager.cc:1397] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 19:30:53.499509 10655 heartbeater.cc:503] Master 127.8.216.190:46575 was elected leader, sending a full tablet report...
I20260807 19:30:53.500231 10821 ts_tablet_manager.cc:1428] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:30:53.501770 10825 tablet_bootstrap.cc:654] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.503095 10825 log.cc:824] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:53.507143 10825 tablet_bootstrap.cc:492] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: No bootstrap required, opened a new log
I20260807 19:30:53.507660 10825 ts_tablet_manager.cc:1397] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:30:53.510139 10822 raft_consensus.cc:348] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.510591 10822 raft_consensus.cc:374] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.510701 10822 raft_consensus.cc:729] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea340443b6ab4c93b68077e48fce246e, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.511339 10822 consensus_queue.cc:260] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.513139 10822 ts_tablet_manager.cc:1428] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260807 19:30:53.517060 10825 raft_consensus.cc:348] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.517463 10825 raft_consensus.cc:374] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.517540 10825 raft_consensus.cc:729] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d076d6991540a494c25da074525e01, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.518095 10825 consensus_queue.cc:260] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.519073 10802 heartbeater.cc:503] Master 127.8.216.190:46575 was elected leader, sending a full tablet report...
I20260807 19:30:53.519640 10825 ts_tablet_manager.cc:1428] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
W20260807 19:30:53.550408 10502 tablet.cc:2369] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:30:53.588506 10656 tablet.cc:2369] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:30:53.611232 10803 tablet.cc:2369] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:30:53.639156 10828 raft_consensus.cc:482] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:53.639519 10828 raft_consensus.cc:504] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.641305 10828 leader_election.cc:288] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130:43251), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:53.650853 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.641743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:51359 (local address 127.8.216.130:43251)
0807 19:30:53.641937 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.641942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.642741 (+ 799us) server_negotiation.cc:407] Connection header received
0807 19:30:53.643020 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.643027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.643114 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.643186 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.644052 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.645253 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.648284 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.649545 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.649724 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.649799 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.650448 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.650577 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:30:53.650846 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.642995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:42767 (local address 127.8.216.131:39161)
0807 19:30:53.643220 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.643232 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.643339 (+ 107us) server_negotiation.cc:407] Connection header received
0807 19:30:53.643555 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.643559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.643617 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.643687 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.644386 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.645408 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.648276 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.649577 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.649723 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.649799 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.650440 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.650577 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:30:53.651113 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.642605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39161 (local address 127.8.216.129:42767)
0807 19:30:53.643249 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.643265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.643383 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.643734 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.643740 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.643771 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.644195 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.644206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.645543 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.645553 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.648084 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.648098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.649854 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.649870 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.650131 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.650291 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.650394 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.650620 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":256,"thread_start_us":312,"threads_started":1}
I20260807 19:30:53.651317 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.641751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43251 (local address 127.8.216.129:51359)
0807 19:30:53.642606 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.642632 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.642806 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.643415 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.643422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.643448 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.643846 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.643861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.645381 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.645385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.648089 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.648099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.649871 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.649876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.650131 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.650291 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.650412 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.650620 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":317,"thread_start_us":437,"threads_started":1}
I20260807 19:30:53.652338 10604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebdfe090cd544cf3900d85679cce7bcf" candidate_uuid: "ea340443b6ab4c93b68077e48fce246e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" is_pre_election: true
I20260807 19:30:53.652357 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebdfe090cd544cf3900d85679cce7bcf" candidate_uuid: "ea340443b6ab4c93b68077e48fce246e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01" is_pre_election: true
I20260807 19:30:53.652981 10604 raft_consensus.cc:2454] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea340443b6ab4c93b68077e48fce246e in term 0.
I20260807 19:30:53.652974 10748 raft_consensus.cc:2454] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea340443b6ab4c93b68077e48fce246e in term 0.
I20260807 19:30:53.653713 10373 leader_election.cc:302] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86d076d6991540a494c25da074525e01, ea340443b6ab4c93b68077e48fce246e; no voters:
I20260807 19:30:53.654049 10828 raft_consensus.cc:2781] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:53.654217 10828 raft_consensus.cc:482] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:53.654321 10828 raft_consensus.cc:3037] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.656674 10828 raft_consensus.cc:504] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.657248 10828 leader_election.cc:288] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [CANDIDATE]: Term 1 election: Requested vote from peers b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130:43251), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:53.657747 10604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebdfe090cd544cf3900d85679cce7bcf" candidate_uuid: "ea340443b6ab4c93b68077e48fce246e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d"
I20260807 19:30:53.657793 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebdfe090cd544cf3900d85679cce7bcf" candidate_uuid: "ea340443b6ab4c93b68077e48fce246e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01"
I20260807 19:30:53.657958 10604 raft_consensus.cc:3037] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.657980 10748 raft_consensus.cc:3037] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.660339 10604 raft_consensus.cc:2454] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea340443b6ab4c93b68077e48fce246e in term 1.
I20260807 19:30:53.660774 10373 leader_election.cc:302] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1bf7c2a3ade47a2b7837491a1a5276d, ea340443b6ab4c93b68077e48fce246e; no voters:
I20260807 19:30:53.660934 10748 raft_consensus.cc:2454] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea340443b6ab4c93b68077e48fce246e in term 1.
I20260807 19:30:53.661067 10828 raft_consensus.cc:2781] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:53.661609 10828 raft_consensus.cc:686] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 1 LEADER]: Becoming Leader. State: Replica: ea340443b6ab4c93b68077e48fce246e, State: Running, Role: LEADER
I20260807 19:30:53.662109 10828 consensus_queue.cc:237] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.666766 10277 catalog_manager.cc:5495] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e reported cstate change: term changed from 0 to 1, leader changed from <none> to ea340443b6ab4c93b68077e48fce246e (127.8.216.129). New cstate: current_term: 1 leader_uuid: "ea340443b6ab4c93b68077e48fce246e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } health_report { overall_health: UNKNOWN } } }
I20260807 19:30:53.701285 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.695327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.8.216.129:32791)
0807 19:30:53.695597 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.695605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.695732 (+ 127us) server_negotiation.cc:407] Connection header received
0807 19:30:53.696170 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.696176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.696268 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.696420 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:53.696803 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.697531 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.698880 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.699302 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.699452 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.699994 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:53.700620 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:53.700803 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.700910 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.701001 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:30:53.709648 10748 raft_consensus.cc:1261] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Refusing update from remote peer ea340443b6ab4c93b68077e48fce246e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:53.709686 10604 raft_consensus.cc:1261] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Refusing update from remote peer ea340443b6ab4c93b68077e48fce246e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:53.710670 10833 consensus_queue.cc:1035] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:53.711043 10828 consensus_queue.cc:1035] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:53.722728 10836 mvcc.cc:205] Tried to move back new op lower bound from 7315992795984986112 to 7315992795803983872. Current Snapshot: MvccSnapshot[applied={T|T < 7315992795984986112}]
I20260807 19:30:53.745136 10277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:53.745471 10277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:53.750334 10277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Sending DeleteTablet for 3 replicas of tablet ebdfe090cd544cf3900d85679cce7bcf
I20260807 19:30:53.751601 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet ebdfe090cd544cf3900d85679cce7bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:53 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:32844
I20260807 19:30:53.752678 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet ebdfe090cd544cf3900d85679cce7bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:53 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40368
I20260807 19:30:53.752828 10850 tablet_replica.cc:330] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: stopping tablet replica
I20260807 19:30:53.753006 10578 tablet_service.cc:1514] Processing DeleteTablet for tablet ebdfe090cd544cf3900d85679cce7bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:53 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47388
I20260807 19:30:53.754122 10852 tablet_replica.cc:330] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: stopping tablet replica
I20260807 19:30:53.754266 10277 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:30:53.754662 10852 raft_consensus.cc:2229] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 19:30:53.754962 10277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:53.755297 10852 raft_consensus.cc:2258] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:53.756212 10851 tablet_replica.cc:330] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: stopping tablet replica
I20260807 19:30:53.756335 10850 raft_consensus.cc:2229] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:30:53.756613 10851 raft_consensus.cc:2229] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:30:53.757014 10850 raft_consensus.cc:2258] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:53.757098 10851 raft_consensus.cc:2258] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:53.758788 10852 ts_tablet_manager.cc:1905] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:53.759433 10851 ts_tablet_manager.cc:1905] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:53.761124 10850 ts_tablet_manager.cc:1905] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:53.765575 10578 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0b3070c30c489594b12d3b02decc69 (DEFAULT_TABLE table=test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.764611 10425 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0b3070c30c489594b12d3b02decc69 (DEFAULT_TABLE table=test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.766044 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0b3070c30c489594b12d3b02decc69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.766168 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0b3070c30c489594b12d3b02decc69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.766176 10851 ts_tablet_manager.cc:1918] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:30:53.766317 10851 log.cc:1196] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/ebdfe090cd544cf3900d85679cce7bcf
I20260807 19:30:53.766891 10851 ts_tablet_manager.cc:1939] T ebdfe090cd544cf3900d85679cce7bcf P 86d076d6991540a494c25da074525e01: Deleting consensus metadata
I20260807 19:30:53.768797 10263 catalog_manager.cc:4841] TS 86d076d6991540a494c25da074525e01 (127.8.216.131:39161): tablet ebdfe090cd544cf3900d85679cce7bcf (table test-table [id=33a63b65a6074527bf2cba44d4a459b9]) successfully deleted
I20260807 19:30:53.769609 10727 tablet_service.cc:1467] Processing CreateTablet for tablet 1d0b3070c30c489594b12d3b02decc69 (DEFAULT_TABLE table=test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:53.770023 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d0b3070c30c489594b12d3b02decc69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.773782 10822 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Bootstrap starting.
I20260807 19:30:53.775951 10822 tablet_bootstrap.cc:654] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.776216 10821 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Bootstrap starting.
I20260807 19:30:53.777125 10825 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Bootstrap starting.
I20260807 19:30:53.778064 10821 tablet_bootstrap.cc:654] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.778892 10825 tablet_bootstrap.cc:654] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:53.780798 10821 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: No bootstrap required, opened a new log
I20260807 19:30:53.781002 10821 ts_tablet_manager.cc:1397] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:30:53.782470 10821 raft_consensus.cc:348] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.782841 10821 raft_consensus.cc:374] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.783144 10821 raft_consensus.cc:729] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bf7c2a3ade47a2b7837491a1a5276d, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.783569 10821 consensus_queue.cc:260] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.784494 10821 ts_tablet_manager.cc:1428] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:53.788504 10826 raft_consensus.cc:482] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:53.788720 10826 raft_consensus.cc:504] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.790967 10826 leader_election.cc:288] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:53.795732 10822 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: No bootstrap required, opened a new log
I20260807 19:30:53.796061 10822 ts_tablet_manager.cc:1397] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260807 19:30:53.798610 10822 raft_consensus.cc:348] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.799024 10822 raft_consensus.cc:374] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.799562 10822 raft_consensus.cc:729] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea340443b6ab4c93b68077e48fce246e, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.800163 10822 consensus_queue.cc:260] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.801872 10822 ts_tablet_manager.cc:1428] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260807 19:30:53.809304 10852 ts_tablet_manager.cc:1918] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:30:53.810694 10852 log.cc:1196] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/ebdfe090cd544cf3900d85679cce7bcf
I20260807 19:30:53.812654 10825 tablet_bootstrap.cc:492] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: No bootstrap required, opened a new log
I20260807 19:30:53.812978 10825 ts_tablet_manager.cc:1397] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.011s
I20260807 19:30:53.815106 10825 raft_consensus.cc:348] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.815418 10825 raft_consensus.cc:374] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:53.815635 10825 raft_consensus.cc:729] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d076d6991540a494c25da074525e01, State: Initialized, Role: FOLLOWER
I20260807 19:30:53.816226 10825 consensus_queue.cc:260] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.818142 10825 ts_tablet_manager.cc:1428] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260807 19:30:53.819355 10852 ts_tablet_manager.cc:1939] T ebdfe090cd544cf3900d85679cce7bcf P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting consensus metadata
I20260807 19:30:53.831357 10850 ts_tablet_manager.cc:1918] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:30:53.831733 10850 log.cc:1196] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/ebdfe090cd544cf3900d85679cce7bcf
I20260807 19:30:53.831729 10263 catalog_manager.cc:4841] TS b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130:43251): tablet ebdfe090cd544cf3900d85679cce7bcf (table test-table [id=33a63b65a6074527bf2cba44d4a459b9]) successfully deleted
I20260807 19:30:53.832892 10850 ts_tablet_manager.cc:1939] T ebdfe090cd544cf3900d85679cce7bcf P ea340443b6ab4c93b68077e48fce246e: Deleting consensus metadata
I20260807 19:30:53.837641 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.802551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:58583 (local address 127.8.216.131:39161)
0807 19:30:53.802786 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.802792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.818139 (+ 15347us) server_negotiation.cc:407] Connection header received
0807 19:30:53.818503 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.818510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.818588 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.818669 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.820707 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.822576 (+ 1869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.833641 (+ 11065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.835556 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.835721 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.835818 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.836910 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.837086 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:30:53.840723 10261 catalog_manager.cc:4841] TS ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791): tablet ebdfe090cd544cf3900d85679cce7bcf (table test-table [id=33a63b65a6074527bf2cba44d4a459b9]) successfully deleted
I20260807 19:30:53.843426 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.791375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:43413 (local address 127.8.216.129:32791)
0807 19:30:53.794022 (+ 2647us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.794032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.810485 (+ 16453us) server_negotiation.cc:407] Connection header received
0807 19:30:53.810723 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.810728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.810814 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.810958 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:53.819890 (+ 8932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.821918 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.833683 (+ 11765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.840688 (+ 7005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.840990 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.841117 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.842569 (+ 1452us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.842843 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2384}
I20260807 19:30:53.845415 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.793183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39161 (local address 127.8.216.130:58583)
0807 19:30:53.818026 (+ 24843us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.818049 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.818302 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.819017 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.819023 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.819052 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.820412 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.820427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.822767 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.822772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.833341 (+ 10569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.833364 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.836289 (+ 2925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.836297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.836679 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.836715 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.837992 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.838188 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24667,"mutex_wait_us":1,"thread_start_us":16296,"threads_started":1}
I20260807 19:30:53.846493 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0b3070c30c489594b12d3b02decc69" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01" is_pre_election: true
I20260807 19:30:53.846951 10748 raft_consensus.cc:2454] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 0.
I20260807 19:30:53.847952 10857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.791766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:32791 (local address 127.8.216.130:43413)
0807 19:30:53.810362 (+ 18596us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.810390 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.810567 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.811218 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.811224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.811279 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:53.819653 (+ 8374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.819671 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.822091 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.822097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.833466 (+ 11369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.833478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.841546 (+ 8068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.841555 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.841866 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.841942 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.842071 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.842297 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17591,"thread_start_us":25595,"threads_started":1}
I20260807 19:30:53.847767 10526 leader_election.cc:302] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86d076d6991540a494c25da074525e01, b1bf7c2a3ade47a2b7837491a1a5276d; no voters:
I20260807 19:30:53.848443 10826 raft_consensus.cc:2781] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:53.848869 10826 raft_consensus.cc:482] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:53.849021 10826 raft_consensus.cc:3037] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.853021 10451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0b3070c30c489594b12d3b02decc69" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea340443b6ab4c93b68077e48fce246e" is_pre_election: true
I20260807 19:30:53.853953 10451 raft_consensus.cc:2454] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 0.
I20260807 19:30:53.854430 10826 raft_consensus.cc:504] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.856014 10451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0b3070c30c489594b12d3b02decc69" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea340443b6ab4c93b68077e48fce246e"
I20260807 19:30:53.856374 10451 raft_consensus.cc:3037] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.857946 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d0b3070c30c489594b12d3b02decc69" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01"
I20260807 19:30:53.858332 10748 raft_consensus.cc:3037] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:53.861953 10451 raft_consensus.cc:2454] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 1.
I20260807 19:30:53.862764 10524 leader_election.cc:302] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1bf7c2a3ade47a2b7837491a1a5276d, ea340443b6ab4c93b68077e48fce246e; no voters:
I20260807 19:30:53.863255 10748 raft_consensus.cc:2454] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 1.
I20260807 19:30:53.863991 10826 leader_election.cc:288] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 election: Requested vote from peers ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:53.864374 10826 raft_consensus.cc:2781] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:53.864692 10826 raft_consensus.cc:686] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 LEADER]: Becoming Leader. State: Replica: b1bf7c2a3ade47a2b7837491a1a5276d, State: Running, Role: LEADER
I20260807 19:30:53.865440 10826 consensus_queue.cc:237] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:53.878902 10277 catalog_manager.cc:5495] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130). New cstate: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:30:53 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131051, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:53.969844 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.947351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.8.216.190:46575)
0807 19:30:53.948972 (+ 1621us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:53.948978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:53.949779 (+ 801us) server_negotiation.cc:407] Connection header received
0807 19:30:53.950015 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:53.950018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:53.950123 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:53.950256 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:53.952006 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.953477 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.955300 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.956040 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.956224 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.961809 (+ 5585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:53.961820 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:53.961828 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:53.961873 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:53.966257 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.967271 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.967287 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.967291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.967455 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:53.967902 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:53.967907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:53.967911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:53.968384 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:53.968628 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:53.969054 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 19:30:53.969353 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1347,"thread_start_us":957,"threads_started":1}
I20260807 19:30:53.970083 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:53.947352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46575 (local address 127.0.0.1:44578)
0807 19:30:53.949647 (+ 2295us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:53.949664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:53.949870 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:53.950277 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:53.950281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:53.951349 (+ 1068us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:53.951711 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.951721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.953665 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.953669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.955142 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:53.955157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:53.956389 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:53.956399 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:53.956849 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:53.957829 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:53.957874 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:53.961323 (+ 3449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:53.966471 (+ 5148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.966487 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.966494 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.967132 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.967583 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:53.967587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:53.967589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:53.967783 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:53.968528 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:53.968534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:53.968714 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 19:30:53.968918 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 19:30:53.969142 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2132,"thread_start_us":1796,"threads_started":1}
I20260807 19:30:53.976812 10277 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44578:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:30:53.978257 10277 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:53.987080 10578 tablet_service.cc:1467] Processing CreateTablet for tablet 8470a3c14ff541038fba6717a2b47fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae52cde4ae84ee48bf8dd4e43b4471c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:53.987159 10727 tablet_service.cc:1467] Processing CreateTablet for tablet 8470a3c14ff541038fba6717a2b47fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae52cde4ae84ee48bf8dd4e43b4471c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:53.987159 10425 tablet_service.cc:1467] Processing CreateTablet for tablet 8470a3c14ff541038fba6717a2b47fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae52cde4ae84ee48bf8dd4e43b4471c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:53.987969 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8470a3c14ff541038fba6717a2b47fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.987985 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8470a3c14ff541038fba6717a2b47fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.988291 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8470a3c14ff541038fba6717a2b47fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:53.998919 10822 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: Bootstrap starting.
I20260807 19:30:53.999329 10821 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: Bootstrap starting.
I20260807 19:30:54.002446 10822 tablet_bootstrap.cc:654] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:54.002730 10825 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: Bootstrap starting.
I20260807 19:30:54.005935 10825 tablet_bootstrap.cc:654] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:54.007007 10821 tablet_bootstrap.cc:654] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:54.009678 10822 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: No bootstrap required, opened a new log
I20260807 19:30:54.010113 10822 ts_tablet_manager.cc:1397] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260807 19:30:54.011616 10822 raft_consensus.cc:348] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.013453 10822 raft_consensus.cc:374] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:54.013635 10822 raft_consensus.cc:729] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea340443b6ab4c93b68077e48fce246e, State: Initialized, Role: FOLLOWER
I20260807 19:30:54.014142 10822 consensus_queue.cc:260] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.015403 10822 ts_tablet_manager.cc:1428] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260807 19:30:54.015012 10846 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.015816 10846 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.017339 10825 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: No bootstrap required, opened a new log
I20260807 19:30:54.017590 10825 ts_tablet_manager.cc:1397] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260807 19:30:54.018249 10821 tablet_bootstrap.cc:492] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: No bootstrap required, opened a new log
I20260807 19:30:54.018555 10821 ts_tablet_manager.cc:1397] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.006s
I20260807 19:30:54.019021 10825 raft_consensus.cc:348] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.019451 10825 raft_consensus.cc:374] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:54.019635 10825 raft_consensus.cc:729] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86d076d6991540a494c25da074525e01, State: Initialized, Role: FOLLOWER
I20260807 19:30:54.019826 10821 raft_consensus.cc:348] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.020246 10821 raft_consensus.cc:374] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:54.020051 10825 consensus_queue.cc:260] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.020452 10821 raft_consensus.cc:729] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bf7c2a3ade47a2b7837491a1a5276d, State: Initialized, Role: FOLLOWER
I20260807 19:30:54.020927 10829 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.021575 10829 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.022209 10821 consensus_queue.cc:260] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.023105 10821 ts_tablet_manager.cc:1428] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 19:30:54.023379 10858 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.024313 10825 ts_tablet_manager.cc:1428] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260807 19:30:54.024405 10858 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.035184 10858 raft_consensus.cc:482] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:54.035610 10858 raft_consensus.cc:504] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.036962 10858 leader_election.cc:288] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:54.037581 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8470a3c14ff541038fba6717a2b47fd2" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01" is_pre_election: true
I20260807 19:30:54.037958 10451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8470a3c14ff541038fba6717a2b47fd2" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea340443b6ab4c93b68077e48fce246e" is_pre_election: true
I20260807 19:30:54.038089 10748 raft_consensus.cc:2454] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 0.
I20260807 19:30:54.038596 10451 raft_consensus.cc:2454] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 0.
I20260807 19:30:54.038829 10526 leader_election.cc:302] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86d076d6991540a494c25da074525e01, b1bf7c2a3ade47a2b7837491a1a5276d; no voters:
I20260807 19:30:54.039273 10858 raft_consensus.cc:2781] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:54.039515 10858 raft_consensus.cc:482] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:54.039714 10858 raft_consensus.cc:3037] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:54.043558 10858 raft_consensus.cc:504] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.044713 10858 leader_election.cc:288] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 election: Requested vote from peers ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791), 86d076d6991540a494c25da074525e01 (127.8.216.131:39161)
I20260807 19:30:54.045167 10451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8470a3c14ff541038fba6717a2b47fd2" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea340443b6ab4c93b68077e48fce246e"
I20260807 19:30:54.045536 10451 raft_consensus.cc:3037] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:54.045502 10748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8470a3c14ff541038fba6717a2b47fd2" candidate_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86d076d6991540a494c25da074525e01"
I20260807 19:30:54.045859 10748 raft_consensus.cc:3037] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:54.049409 10748 raft_consensus.cc:2454] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 1.
I20260807 19:30:54.049942 10451 raft_consensus.cc:2454] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1bf7c2a3ade47a2b7837491a1a5276d in term 1.
I20260807 19:30:54.050024 10526 leader_election.cc:302] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86d076d6991540a494c25da074525e01, b1bf7c2a3ade47a2b7837491a1a5276d; no voters:
I20260807 19:30:54.050601 10858 raft_consensus.cc:2781] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:54.050875 10858 raft_consensus.cc:686] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 LEADER]: Becoming Leader. State: Replica: b1bf7c2a3ade47a2b7837491a1a5276d, State: Running, Role: LEADER
I20260807 19:30:54.051249 10858 consensus_queue.cc:237] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } }
I20260807 19:30:54.054690 10826 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: TxnStatusTablet state changed. Reason: New leader b1bf7c2a3ade47a2b7837491a1a5276d. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.055164 10826 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:54.055939 10277 catalog_manager.cc:5495] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d reported cstate change: term changed from 0 to 1, leader changed from <none> to b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130). New cstate: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } health_report { overall_health: UNKNOWN } } }
I20260807 19:30:54.057595 10874 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:30:54.058038 10874 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:30:54.095515 10867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.084317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43251 (local address 127.0.0.1:47404)
0807 19:30:54.084561 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:54.084581 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:54.084816 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:54.085563 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:54.085568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:54.085582 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:30:54.086004 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:54.086023 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.088466 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.088472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:54.090770 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:54.090788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.092443 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.092450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:54.093076 (+ 626us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.093182 (+ 106us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:30:54.094582 (+ 1400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:30:54.094614 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:30:54.094731 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:30:54.094985 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:30:54.095618 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.084116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.8.216.130:43251)
0807 19:30:54.084408 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.084416 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.084661 (+ 245us) server_negotiation.cc:407] Connection header received
0807 19:30:54.085088 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.085094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.085272 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.085440 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:54.086216 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.088278 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.090948 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.092060 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.092276 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.093486 (+ 1210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:54.094414 (+ 928us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:54.094637 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.094774 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.094888 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:30:54.103240 10748 raft_consensus.cc:1261] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Refusing update from remote peer b1bf7c2a3ade47a2b7837491a1a5276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:54.103986 10451 raft_consensus.cc:1261] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Refusing update from remote peer b1bf7c2a3ade47a2b7837491a1a5276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:54.104430 10858 consensus_queue.cc:1035] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:54.105738 10858 consensus_queue.cc:1035] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:54.120774 10829 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: TxnStatusTablet state changed. Reason: New leader b1bf7c2a3ade47a2b7837491a1a5276d. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.121186 10829 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.126741 10858 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.127357 10858 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:54.128238 10837 mvcc.cc:205] Tried to move back new op lower bound from 7315992797594267648 to 7315992797396197376. Current Snapshot: MvccSnapshot[applied={T|T < 7315992797594267648 or (T in {7315992797594267648})}]
I20260807 19:30:54.129222 10826 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.129601 10826 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P b1bf7c2a3ade47a2b7837491a1a5276d: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:54.130894 10846 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: TxnStatusTablet state changed. Reason: New leader b1bf7c2a3ade47a2b7837491a1a5276d. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.132380 10846 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.134938 10846 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.135331 10846 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P ea340443b6ab4c93b68077e48fce246e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.138953 10829 tablet_replica.cc:439] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 } } peers { permanent_uuid: "b1bf7c2a3ade47a2b7837491a1a5276d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43251 } } peers { permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 } } }
I20260807 19:30:54.139312 10829 tablet_replica.cc:442] T 8470a3c14ff541038fba6717a2b47fd2 P 86d076d6991540a494c25da074525e01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:54.151826 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.140400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.8.216.130:43251)
0807 19:30:54.140639 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.140656 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.140676 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:30:54.140748 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.140753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.140876 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.140982 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:54.141842 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.143628 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.147885 (+ 4257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.149044 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.149180 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.150130 (+ 950us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:54.150277 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:54.150388 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.150993 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.151108 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:30:54.176890 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.167026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:45481 (local address 127.8.216.130:43251)
0807 19:30:54.167403 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.167413 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.168893 (+ 1480us) server_negotiation.cc:407] Connection header received
0807 19:30:54.169182 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.169188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.169286 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.169392 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:54.170708 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.171878 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.174396 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.175245 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.175432 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.175983 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:54.176110 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:54.176261 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.176424 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.176497 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:30:54.177268 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.166914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43251 (local address 127.8.216.130:45481)
0807 19:30:54.168747 (+ 1833us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:54.168774 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:54.169028 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:54.169418 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:54.169423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:54.169434 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:30:54.170450 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:54.170465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.172037 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.172044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:54.174070 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:54.174083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.175595 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.175601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:54.175720 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.175774 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:30:54.176270 (+ 496us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:30:54.176293 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:30:54.176394 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 19:30:54.176543 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"mutex_wait_us":3,"thread_start_us":1201,"threads_started":1}
I20260807 19:30:54.194160 10747 raft_consensus.cc:1261] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Refusing update from remote peer b1bf7c2a3ade47a2b7837491a1a5276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:54.195444 10858 consensus_queue.cc:1035] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "86d076d6991540a494c25da074525e01" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:54.201083 10451 raft_consensus.cc:1261] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Refusing update from remote peer b1bf7c2a3ade47a2b7837491a1a5276d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:54.202474 10858 consensus_queue.cc:1035] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea340443b6ab4c93b68077e48fce246e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:54.350086 10277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:54.350600 10277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:54.356031 10277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e15ef756590426e86d4c7aedb8b8014: Sending DeleteTablet for 3 replicas of tablet 1d0b3070c30c489594b12d3b02decc69
I20260807 19:30:54.357299 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0b3070c30c489594b12d3b02decc69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:32844
I20260807 19:30:54.357967 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0b3070c30c489594b12d3b02decc69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40368
I20260807 19:30:54.358325 10577 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d0b3070c30c489594b12d3b02decc69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47388
I20260807 19:30:54.373199 10888 tablet_replica.cc:330] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: stopping tablet replica
I20260807 19:30:54.373603 10888 raft_consensus.cc:2229] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:30:54.374184 10888 raft_consensus.cc:2258] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:54.376271 10887 tablet_replica.cc:330] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: stopping tablet replica
I20260807 19:30:54.376637 10887 raft_consensus.cc:2229] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:30:54.377008 10888 ts_tablet_manager.cc:1905] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:54.377029 10887 raft_consensus.cc:2258] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:54.378574 10887 ts_tablet_manager.cc:1905] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:54.385869 10889 tablet_replica.cc:330] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: stopping tablet replica
I20260807 19:30:54.386340 10889 raft_consensus.cc:2229] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:30:54.387590 10889 raft_consensus.cc:2258] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:54.397720 10889 ts_tablet_manager.cc:1905] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:54.402659 10888 ts_tablet_manager.cc:1918] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:54.403041 10888 log.cc:1196] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/1d0b3070c30c489594b12d3b02decc69
I20260807 19:30:54.404160 10888 ts_tablet_manager.cc:1939] T 1d0b3070c30c489594b12d3b02decc69 P 86d076d6991540a494c25da074525e01: Deleting consensus metadata
I20260807 19:30:54.407245 10263 catalog_manager.cc:4841] TS 86d076d6991540a494c25da074525e01 (127.8.216.131:39161): tablet 1d0b3070c30c489594b12d3b02decc69 (table test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]) successfully deleted
I20260807 19:30:54.417240 10887 ts_tablet_manager.cc:1918] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:54.417506 10887 log.cc:1196] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/1d0b3070c30c489594b12d3b02decc69
I20260807 19:30:54.418623 10887 ts_tablet_manager.cc:1939] T 1d0b3070c30c489594b12d3b02decc69 P ea340443b6ab4c93b68077e48fce246e: Deleting consensus metadata
I20260807 19:30:54.420192 10261 catalog_manager.cc:4841] TS ea340443b6ab4c93b68077e48fce246e (127.8.216.129:32791): tablet 1d0b3070c30c489594b12d3b02decc69 (table test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]) successfully deleted
I20260807 19:30:54.422760 10889 ts_tablet_manager.cc:1918] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:54.423064 10889 log.cc:1196] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/1d0b3070c30c489594b12d3b02decc69
I20260807 19:30:54.423830 10889 ts_tablet_manager.cc:1939] T 1d0b3070c30c489594b12d3b02decc69 P b1bf7c2a3ade47a2b7837491a1a5276d: Deleting consensus metadata
I20260807 19:30:54.425220 10263 catalog_manager.cc:4841] TS b1bf7c2a3ade47a2b7837491a1a5276d (127.8.216.130:43251): tablet 1d0b3070c30c489594b12d3b02decc69 (table test-table [id=0b5a296ed70f4d02886c99de0f9bb5e2]) successfully deleted
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:30:54.451539 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.382531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.8.216.129:32791)
0807 19:30:54.404672 (+ 22141us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.404680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.416275 (+ 11595us) server_negotiation.cc:407] Connection header received
0807 19:30:54.416702 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.416709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.416952 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.417051 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:54.421409 (+ 4358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.423174 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.429984 (+ 6810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.431237 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.431387 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.440012 (+ 8625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:54.440036 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:54.440058 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:54.440107 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:54.446718 (+ 6611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.448205 (+ 1487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.448216 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.448221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.448432 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.448971 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.448976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.448980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.449592 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:54.449920 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.450749 (+ 829us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.450952 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21526,"thread_start_us":21132,"threads_started":1}
W20260807 19:30:54.453802 10495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32866
I20260807 19:30:54.480018 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.458496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.8.216.190:46575)
0807 19:30:54.458754 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.458775 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.461255 (+ 2480us) server_negotiation.cc:407] Connection header received
0807 19:30:54.461618 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.461624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.461728 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.461837 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:54.464102 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.465010 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.466931 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.467741 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.467890 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.472010 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:54.472015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:54.472018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:54.472056 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:54.476315 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.477254 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.477261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.477264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.477448 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.477921 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.477930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.477933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.478328 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:54.478688 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.479084 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.479484 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:30:54.480973 10277 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44586
I20260807 19:30:54.483981 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10352
I20260807 19:30:54.506692 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.485725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.8.216.129:32791)
0807 19:30:54.485959 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.485968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.486426 (+ 458us) server_negotiation.cc:407] Connection header received
0807 19:30:54.486704 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.486708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.486775 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.486889 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:54.488658 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.489466 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.491408 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.492423 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.492577 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.496014 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:54.496019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:54.496023 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:54.496061 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:54.500615 (+ 4554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.501557 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.501564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.501567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.501752 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.502535 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.502545 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.502550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.503210 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:54.503708 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.505781 (+ 2073us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.506072 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 19:30:54.508308 10495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32882
W20260807 19:30:54.509156 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10352: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:30:54.509531 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10352
I20260807 19:30:54.543203 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10506
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:30:54.560698 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.544042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.8.216.130:43251)
0807 19:30:54.544354 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.544360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.544381 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:30:54.544559 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.544565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.544637 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.544749 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:54.546788 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.547578 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.549199 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.549674 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.549821 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.554229 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:54.554241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:54.554257 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:54.554320 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:54.557666 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.558379 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.558387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.558389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.558541 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.558953 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.558957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.558959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.559354 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:54.559760 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.560047 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.560342 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
W20260807 19:30:54.561395 10649 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47420
W20260807 19:30:54.562126 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:30:54.562292 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10506
I20260807 19:30:54.584568 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10660
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131053, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:30:54.600080 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:54.585345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40370 (local address 127.8.216.131:39161)
0807 19:30:54.585852 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:54.585858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:54.585876 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:30:54.585933 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:54.585937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:54.585997 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:54.586092 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:54.587434 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.588177 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.589346 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:54.589764 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:54.589859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:54.594210 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:54.594226 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:54.594243 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:54.594287 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:54.597401 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.598181 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.598188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.598190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.598322 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:54.598734 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:54.598737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:54.598739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:54.599095 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:54.599321 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:54.599605 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:30:54.599801 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":304,"threads_started":1}
W20260807 19:30:54.600842 10794 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40370
W20260807 19:30:54.601532 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10660: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:30:54.601693 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10660
I20260807 19:30:54.621953 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10242
I20260807 19:30:54.623158 10318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:30:54.740302 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10242
2026-08-07T19:30:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4946 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10901](info): set up 2 sockets
Aug 07 19:30:54 dist-test-slave-8w9v krb5kdc[10901](info): commencing operation
krb5kdc: starting...
W20260807 19:30:55.903697 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:30:56 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:30:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:30:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:30:56.113986 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:34693
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:34693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:56.451727 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:56.452106 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:56.452349 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:56.459836 10916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:30:56.460011 10916 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:30:56.460110 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:56.460242 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:56.460337 10916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:30:56.460462 10916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:30:56.470759 10916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:34693
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:34693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:56.472214 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:56.473546 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:56.485965 10922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:56.485903 10923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:56.485935 10925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:56.487192 10916 server_base.cc:1034] running on GCE node
I20260807 19:30:56.487980 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:56.489527 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:56.490764 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786131056490693 us; error 41 us; skew 500 ppm
I20260807 19:30:56.491128 10916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:56 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131056, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:56.494477 10916 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:30:56.495980 10916 webserver.cc:460] Webserver started at http://127.8.216.190:43767/ using document root <none> and password file <none>
I20260807 19:30:56.496771 10916 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:56.496908 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:56.497237 10916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:56.500473 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "5aab6f99998e49a78919b2b5e9b29692"
format_stamp: "Formatted at 2026-08-07 19:30:56 on dist-test-slave-8w9v"
server_key: "e822cb9902b2f8b1d172603aa6440c00"
server_key_iv: "abcbd4b8cffde760958b09dd33bb5057"
server_key_version: "encryptionkey@0"
I20260807 19:30:56.501287 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "5aab6f99998e49a78919b2b5e9b29692"
format_stamp: "Formatted at 2026-08-07 19:30:56 on dist-test-slave-8w9v"
server_key: "e822cb9902b2f8b1d172603aa6440c00"
server_key_iv: "abcbd4b8cffde760958b09dd33bb5057"
server_key_version: "encryptionkey@0"
I20260807 19:30:56.506807 10916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:30:56.510324 10933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:56.511801 10916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:30:56.512008 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "5aab6f99998e49a78919b2b5e9b29692"
format_stamp: "Formatted at 2026-08-07 19:30:56 on dist-test-slave-8w9v"
server_key: "e822cb9902b2f8b1d172603aa6440c00"
server_key_iv: "abcbd4b8cffde760958b09dd33bb5057"
server_key_version: "encryptionkey@0"
I20260807 19:30:56.512311 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:56.529393 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:56.534235 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:56.534582 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:56.558372 10916 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:34693
I20260807 19:30:56.558396 10996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:34693 every 8 connection(s)
I20260807 19:30:56.559826 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:30:56.563814 10997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:56.566147 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 10916
I20260807 19:30:56.566413 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:30:56.567219 9058 external_mini_cluster.cc:1428] Setting key c208e1b32898d29bfb584a108c6e262a
I20260807 19:30:56.577587 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Bootstrap starting.
I20260807 19:30:56.581863 10997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Neither blocks nor log segments found. Creating new log.
Aug 07 19:30:56 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:56.583159 10997 log.cc:824] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:56.586910 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: No bootstrap required, opened a new log
I20260807 19:30:56.593058 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:56.569411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.8.216.190:34693)
0807 19:30:56.570289 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:56.570302 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:56.570347 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:30:56.572104 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:56.572206 (+ 102us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:56.572591 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:56.572901 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:56.574868 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:56.576372 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:56.578160 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:56.578895 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:56.579038 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:56.584014 (+ 4976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:56.584022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:56.584039 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:56.584086 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:56.588223 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:56.588942 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:56.588951 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:56.588961 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:56.589195 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:56.589564 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:56.589568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:56.589572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:56.590396 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:56.590723 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:56.591095 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:30:56.591587 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":439,"threads_started":1}
I20260807 19:30:56.597467 10997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } }
I20260807 19:30:56.597841 10997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:56.597971 10997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aab6f99998e49a78919b2b5e9b29692, State: Initialized, Role: FOLLOWER
I20260807 19:30:56.598572 10997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } }
I20260807 19:30:56.598814 10997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:30:56.598980 10997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:30:56.599152 10997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:56.601842 10997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } }
I20260807 19:30:56.602370 10997 leader_election.cc:302] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5aab6f99998e49a78919b2b5e9b29692; no voters:
I20260807 19:30:56.602985 10997 leader_election.cc:288] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:30:56.603138 11002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:56.603963 11002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [term 1 LEADER]: Becoming Leader. State: Replica: 5aab6f99998e49a78919b2b5e9b29692, State: Running, Role: LEADER
I20260807 19:30:56.604571 11002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } }
I20260807 19:30:56.605232 10997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:30:56.608100 11003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5aab6f99998e49a78919b2b5e9b29692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } } }
I20260807 19:30:56.608824 11004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5aab6f99998e49a78919b2b5e9b29692. Latest consensus state: current_term: 1 leader_uuid: "5aab6f99998e49a78919b2b5e9b29692" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aab6f99998e49a78919b2b5e9b29692" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34693 } } }
I20260807 19:30:56.609125 11003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:56.609228 11004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692 [sys.catalog]: This master's current role is: LEADER
I20260807 19:30:56.610441 11012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:30:56.616714 11012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:30:56.626108 11012 catalog_manager.cc:1347] Generated new cluster ID: 9f613706cf3543eaad3410dda66e7598
I20260807 19:30:56.626238 11012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:30:56.637532 11012 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:30:56.639498 11012 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:30:56.660889 11012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Generated new TSK 0
I20260807 19:30:56.661866 11012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:30:56.724099 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34693
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:57.067608 11026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:57.067893 11026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:57.067994 11026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:57.075301 11026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:57.075433 11026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:57.075519 11026 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:30:57.075565 11026 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:57.075636 11026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:30:57.085790 11026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:57.087069 11026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:57.088470 11026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:57.101759 11032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:57.101835 11033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:57.101792 11035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:57.102803 11026 server_base.cc:1034] running on GCE node
I20260807 19:30:57.103433 11026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:57.104470 11026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:57.105711 11026 hybrid_clock.cc:648] HybridClock initialized: now 1786131057105680 us; error 46 us; skew 500 ppm
I20260807 19:30:57.106096 11026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:57 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:57.109438 11026 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:30:57.111255 11026 webserver.cc:460] Webserver started at http://127.8.216.129:35523/ using document root <none> and password file <none>
I20260807 19:30:57.112017 11026 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:57.112195 11026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:57.112533 11026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:57.115494 11026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "d9b241d2bd774855a90767d7c799ee02"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "ef6f68ca8bb251398dd3e0e50d39cca7"
server_key_iv: "f96b534ef6b90740501405f7747332c6"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.116281 11026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "d9b241d2bd774855a90767d7c799ee02"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "ef6f68ca8bb251398dd3e0e50d39cca7"
server_key_iv: "f96b534ef6b90740501405f7747332c6"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.121692 11026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:30:57.125212 11042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:57.126670 11026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:30:57.126902 11026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "d9b241d2bd774855a90767d7c799ee02"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "ef6f68ca8bb251398dd3e0e50d39cca7"
server_key_iv: "f96b534ef6b90740501405f7747332c6"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.127138 11026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:57.167485 11026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:57.171512 11026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:57.171846 11026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:57.174561 11026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:57.174702 11026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:57.174840 11026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:57.174937 11026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:57 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:57.204155 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.175303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60673 (local address 127.8.216.190:34693)
0807 19:30:57.175913 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:57.175923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:57.177260 (+ 1337us) server_negotiation.cc:407] Connection header received
0807 19:30:57.179339 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:57.179344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:57.179450 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:57.179562 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:57.181652 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.182984 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.185669 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.186480 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.186641 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.192346 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:57.192355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:57.192360 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:57.192416 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:57.200040 (+ 7624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.201359 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.201365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.201368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.201568 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.202187 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.202192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.202196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.202499 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:57.202667 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:57.203490 (+ 823us) server_negotiation.cc:299] Negotiation successful
0807 19:30:57.203739 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":364,"threads_started":1}
I20260807 19:30:57.205286 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.175997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.129:60673)
0807 19:30:57.177073 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:57.177117 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:57.179052 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:57.180080 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:57.180101 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:57.180857 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:57.181438 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.181467 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.183169 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.183180 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.185444 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.185464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.186785 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.186796 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.187171 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.188030 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:57.188072 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:57.191975 (+ 3903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:57.200312 (+ 8337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.200320 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.200339 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.201167 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.201745 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.201756 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.201762 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.202025 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.202638 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:57.202647 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:57.203071 (+ 424us) client_negotiation.cc:769] Sending connection context
0807 19:30:57.203350 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 19:30:57.203921 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":494,"threads_started":1}
I20260807 19:30:57.232986 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.216324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.129:36469)
0807 19:30:57.217061 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:57.217086 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:57.217253 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:57.217653 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:57.217657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:57.218312 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:57.218683 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.218695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.220163 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.220171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.222038 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.222055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.222883 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.222889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.223059 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.223747 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:57.223765 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:57.224944 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:57.229577 (+ 4633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.229582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.229590 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.230372 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.230928 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.230932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.230935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.231061 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.231777 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:57.231785 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:57.232041 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 19:30:57.232218 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:30:57.232613 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":409,"threads_started":1}
I20260807 19:30:57.233672 11026 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37521
I20260807 19:30:57.233695 11176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37521 every 8 connection(s)
I20260807 19:30:57.233824 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.216271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36469 (local address 127.8.216.190:34693)
0807 19:30:57.216459 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:57.216464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:57.217160 (+ 696us) server_negotiation.cc:407] Connection header received
0807 19:30:57.217412 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:57.217420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:57.217507 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:57.217621 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:57.218921 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.219917 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.222200 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.222666 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.222818 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.225220 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:57.225228 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:57.225233 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:57.225274 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:57.229375 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.230588 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.230595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.230599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.230799 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.231211 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.231215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.231218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.231620 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:57.231829 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:57.233344 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0807 19:30:57.233569 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:30:57.235129 11026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:30:57.236537 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11026
I20260807 19:30:57.236768 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:30:57.237418 9058 external_mini_cluster.cc:1428] Setting key c54542e0a1987b13a7f9cacf2713e68d
I20260807 19:30:57.240499 11178 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34693
I20260807 19:30:57.240738 11178 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:57.241398 11178 heartbeater.cc:511] Master 127.8.216.190:34693 requested a full tablet report, sending...
I20260807 19:30:57.243664 10952 ts_manager.cc:194] Registered new tserver with Master: d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521)
I20260807 19:30:57.245944 10952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:60673
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:30:57.287660 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34693
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:57.624145 11182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:57.624465 11182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:57.624655 11182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:57.631965 11182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:57.632071 11182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:57.632223 11182 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:30:57.632288 11182 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:57.632362 11182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:30:57.642243 11182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:57.643504 11182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:57.644846 11182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:57.658272 11188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:57.658311 11187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:57.658228 11190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:57.659051 11182 server_base.cc:1034] running on GCE node
I20260807 19:30:57.659737 11182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:57.660769 11182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:57.662003 11182 hybrid_clock.cc:648] HybridClock initialized: now 1786131057661943 us; error 61 us; skew 500 ppm
I20260807 19:30:57.662288 11182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:57 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:57.665423 11182 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:30:57.666952 11182 webserver.cc:460] Webserver started at http://127.8.216.130:42109/ using document root <none> and password file <none>
I20260807 19:30:57.667680 11182 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:57.667783 11182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:57.668046 11182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:57.671051 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "fa460ca1ce494177b19e27c8b118dffd"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "43f19aef6e276adefa2be2cfc0c82adc"
server_key_iv: "40c70c43c49f08c2738842929e91dc96"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.671766 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "fa460ca1ce494177b19e27c8b118dffd"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "43f19aef6e276adefa2be2cfc0c82adc"
server_key_iv: "40c70c43c49f08c2738842929e91dc96"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.677219 11182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 19:30:57.680847 11197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:57.682233 11182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:57.682439 11182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "fa460ca1ce494177b19e27c8b118dffd"
format_stamp: "Formatted at 2026-08-07 19:30:57 on dist-test-slave-8w9v"
server_key: "43f19aef6e276adefa2be2cfc0c82adc"
server_key_iv: "40c70c43c49f08c2738842929e91dc96"
server_key_version: "encryptionkey@0"
I20260807 19:30:57.682662 11182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:57.711958 11182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:57.715982 11182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:57.716339 11182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:57.719249 11182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:57.719403 11182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:57.719542 11182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:57.719642 11182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:57 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131057, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:57.746987 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.720628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.130:50035)
0807 19:30:57.721543 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:57.721587 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:57.723466 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:57.724620 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:57.724636 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:57.725628 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:57.726761 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.726795 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.728319 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.728327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.730461 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.730482 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.731797 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.731806 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.732181 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.733201 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:57.733247 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:57.737159 (+ 3912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:57.740897 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.740911 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.740936 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.741831 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.742351 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.742357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.742361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.742697 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.743574 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:57.743581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:57.744227 (+ 646us) client_negotiation.cc:769] Sending connection context
0807 19:30:57.744604 (+ 377us) client_negotiation.cc:240] Negotiation successful
0807 19:30:57.745261 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":402,"threads_started":1}
I20260807 19:30:57.748590 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.719991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:50035 (local address 127.8.216.190:34693)
0807 19:30:57.720222 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:57.720229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:57.721741 (+ 1512us) server_negotiation.cc:407] Connection header received
0807 19:30:57.723821 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:57.723827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:57.723975 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:57.724139 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:57.727030 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.728070 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.730659 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.731508 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.731658 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.737447 (+ 5789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:57.737453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:57.737456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:57.737494 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:57.740708 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.742022 (+ 1314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.742031 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.742037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.742215 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.742876 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.742882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.742886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.743323 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:57.743493 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:57.747833 (+ 4340us) server_negotiation.cc:299] Negotiation successful
0807 19:30:57.748188 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:30:57.774641 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.758587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:56515 (local address 127.8.216.190:34693)
0807 19:30:57.758865 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:57.758876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:57.759483 (+ 607us) server_negotiation.cc:407] Connection header received
0807 19:30:57.759707 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:57.759715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:57.759826 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:57.759978 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:57.761048 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.762540 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.763813 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.764642 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.764794 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.767143 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:57.767151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:57.767158 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:57.767208 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:57.770637 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.771657 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.771665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.771668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.771852 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:57.772472 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:57.772479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:57.772482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:57.772998 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:57.773163 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:57.774114 (+ 951us) server_negotiation.cc:299] Negotiation successful
0807 19:30:57.774400 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:30:57.774845 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:57.758655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.130:56515)
0807 19:30:57.759359 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:57.759386 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:57.759560 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:57.760008 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:57.760011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:57.760590 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:57.760887 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.760896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.762685 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.762689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.763671 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:57.763681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:57.764837 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:57.764841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:57.764966 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:57.765664 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:57.765684 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:57.766935 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:57.770797 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.770801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.770803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.771435 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.771995 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:57.772000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:57.772003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:57.772303 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:57.773136 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:57.773141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:57.773459 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 19:30:57.773663 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 19:30:57.774166 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":404,"threads_started":1}
I20260807 19:30:57.779307 11182 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:40579
I20260807 19:30:57.779345 11321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:40579 every 8 connection(s)
I20260807 19:30:57.780733 11182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:30:57.782823 11322 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34693
I20260807 19:30:57.783073 11322 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:57.783741 11322 heartbeater.cc:511] Master 127.8.216.190:34693 requested a full tablet report, sending...
I20260807 19:30:57.785125 10952 ts_manager.cc:194] Registered new tserver with Master: fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579)
I20260807 19:30:57.786027 10952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:50035
I20260807 19:30:57.790971 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11182
I20260807 19:30:57.791178 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:30:57.791616 9058 external_mini_cluster.cc:1428] Setting key 69dbb0c5440d40f4d001c8e5eae200f6
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:30:57.839339 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34693
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:30:58.178795 11326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:30:58.179090 11326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:30:58.179193 11326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:30:58.186643 11326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:30:58.186759 11326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:30:58.186847 11326 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:30:58.186902 11326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:30:58.187058 11326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:30:58.197190 11326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:34693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:30:58.198552 11326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:30:58.199867 11326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:30:58.213486 11331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:58.213573 11334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:30:58.213651 11332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:30:58.214382 11326 server_base.cc:1034] running on GCE node
I20260807 19:30:58.215073 11326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:30:58.216190 11326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:30:58.217434 11326 hybrid_clock.cc:648] HybridClock initialized: now 1786131058217366 us; error 56 us; skew 500 ppm
I20260807 19:30:58.217847 11326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:30:58 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131058, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:30:58.221323 11326 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:30:58.222966 11326 webserver.cc:460] Webserver started at http://127.8.216.131:43955/ using document root <none> and password file <none>
I20260807 19:30:58.223681 11326 fs_manager.cc:359] Metadata directory not provided
I20260807 19:30:58.223834 11326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:30:58.224223 11326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:30:58.227152 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "31ba61a198af4ef08cd63f337ee0937c"
format_stamp: "Formatted at 2026-08-07 19:30:58 on dist-test-slave-8w9v"
server_key: "65819977e1b5b846b6511aa81390b365"
server_key_iv: "8897a562d107a932f57a3eaf06ba0725"
server_key_version: "encryptionkey@0"
I20260807 19:30:58.227924 11326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "31ba61a198af4ef08cd63f337ee0937c"
format_stamp: "Formatted at 2026-08-07 19:30:58 on dist-test-slave-8w9v"
server_key: "65819977e1b5b846b6511aa81390b365"
server_key_iv: "8897a562d107a932f57a3eaf06ba0725"
server_key_version: "encryptionkey@0"
I20260807 19:30:58.233368 11326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:30:58.237023 11341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:58.238337 11326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:30:58.238533 11326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "31ba61a198af4ef08cd63f337ee0937c"
format_stamp: "Formatted at 2026-08-07 19:30:58 on dist-test-slave-8w9v"
server_key: "65819977e1b5b846b6511aa81390b365"
server_key_iv: "8897a562d107a932f57a3eaf06ba0725"
server_key_version: "encryptionkey@0"
I20260807 19:30:58.238739 11326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:30:58.249717 11178 heartbeater.cc:503] Master 127.8.216.190:34693 was elected leader, sending a full tablet report...
I20260807 19:30:58.252067 11326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:30:58.256840 11326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:30:58.257089 11326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:30:58.259931 11326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:30:58.260061 11326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:30:58.260234 11326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:30:58.260329 11326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:30:58 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131058, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:58.284957 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.260434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:43365 (local address 127.8.216.190:34693)
0807 19:30:58.260639 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.260648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.262148 (+ 1500us) server_negotiation.cc:407] Connection header received
0807 19:30:58.264211 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.264217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.264318 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.264445 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:58.266739 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.267844 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.270715 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.271142 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.271283 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.277017 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:58.277023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:58.277026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:58.277064 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:58.281024 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.282134 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.282142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.282147 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.282360 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.283021 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.283024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.283026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.283328 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:58.283506 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.284315 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.284559 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:30:58.286777 11352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.261140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.131:43365)
0807 19:30:58.261978 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.262024 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.263882 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.264690 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.264701 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.265525 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:58.266519 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.266551 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.268013 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.268017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.270507 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.270522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.271394 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.271402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.271623 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.272523 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:58.272556 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:58.276665 (+ 4109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:58.281201 (+ 4536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.281218 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.281232 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.281949 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.282493 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.282501 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.282507 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.282860 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.283478 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:58.283486 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:58.283883 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.284184 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.284828 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":345,"threads_started":1}
I20260807 19:30:58.309990 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.294345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:51079 (local address 127.8.216.190:34693)
0807 19:30:58.294543 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.294549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.295211 (+ 662us) server_negotiation.cc:407] Connection header received
0807 19:30:58.295517 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.295524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.295633 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.295717 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:58.297425 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.298467 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.299785 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.300297 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.300443 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.302539 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:58.302559 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:58.302565 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:58.302611 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:58.306467 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.307647 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.307654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.307660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.307814 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.308336 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.308343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.308347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.308740 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:58.308924 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.309325 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.309588 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:30:58.310123 11436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.294390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.8.216.131:51079)
0807 19:30:58.295096 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.295112 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.295297 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.295938 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.295941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.296679 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:58.297270 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.297281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.298608 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.298613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.299628 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.299639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.300519 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.300526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.300698 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.301417 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:58.301434 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:58.302309 (+ 875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:58.306671 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.306677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.306692 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.307514 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.307945 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.307953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.307959 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.308188 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.308882 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:58.308886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:58.309082 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.309229 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.309603 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":458,"threads_started":1}
I20260807 19:30:58.312919 11326 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:43945
I20260807 19:30:58.312927 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:43945 every 8 connection(s)
I20260807 19:30:58.314538 11326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:30:58.316628 11475 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34693
I20260807 19:30:58.316879 11475 heartbeater.cc:464] Registering TS with master...
I20260807 19:30:58.317529 11475 heartbeater.cc:511] Master 127.8.216.190:34693 requested a full tablet report, sending...
I20260807 19:30:58.318996 10952 ts_manager.cc:194] Registered new tserver with Master: 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:30:58.319815 10952 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:43365
I20260807 19:30:58.320466 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11326
I20260807 19:30:58.320668 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:30:58.321110 9058 external_mini_cluster.cc:1428] Setting key 4fabb35dcb9f926c9c7b308239ba994f
I20260807 19:30:58.323923 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:30:58 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:30:58 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:58.373658 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.360079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.8.216.190:34693)
0807 19:30:58.360358 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.360365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.360514 (+ 149us) server_negotiation.cc:407] Connection header received
0807 19:30:58.360675 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.360679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.360748 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.360840 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:58.362223 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.362973 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.364290 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.364700 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.364798 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.368676 (+ 3878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:58.368682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:58.368685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:58.368720 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:58.371228 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.371828 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.371832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.371834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.371967 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.372371 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.372374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.372376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.372652 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:58.372842 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.373085 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.373260 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 19:30:58.377785 10952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:30:58.384902 10952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:58.410569 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.398158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:43945 (local address 127.0.0.1:43936)
0807 19:30:58.400614 (+ 2456us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.400649 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.400842 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.402086 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.402095 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.402125 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:58.402592 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.402604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.404998 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.405004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.407728 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.407743 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.409434 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.409441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.409632 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.409691 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.409848 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.410098 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2205,"mutex_wait_us":92,"thread_start_us":2042,"threads_started":1}
I20260807 19:30:58.410665 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.398434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.8.216.131:43945)
0807 19:30:58.401411 (+ 2977us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.401420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.401455 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:30:58.401563 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.401572 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.401841 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.402842 (+ 1001us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:58.403151 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.404855 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.407944 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.409239 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.409405 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.409638 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.410018 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.410188 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2777,"thread_start_us":2565,"threads_started":1}
I20260807 19:30:58.411751 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.398789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.8.216.130:40579)
0807 19:30:58.399972 (+ 1183us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.399986 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.401594 (+ 1608us) server_negotiation.cc:407] Connection header received
0807 19:30:58.401888 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.401897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.402122 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.402303 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:58.403298 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.404857 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.408034 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.409290 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.409425 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.409620 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.411291 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.411485 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":762,"threads_started":1}
I20260807 19:30:58.411938 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.401599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.0.0.1:40148)
0807 19:30:58.403435 (+ 1836us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.403449 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.403595 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.404154 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.404158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.404182 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:58.404497 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.404506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.406365 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.406371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.408650 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.408666 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.411140 (+ 2474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.411149 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.411327 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.411358 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.411442 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.411564 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"mutex_wait_us":25,"thread_start_us":1599,"threads_started":1}
I20260807 19:30:58.412634 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.398227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:40579 (local address 127.0.0.1:40104)
0807 19:30:58.401489 (+ 3262us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.401508 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.401695 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.402577 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.402582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.402606 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:58.403070 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.403084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.405396 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.405401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.407809 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.407825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.409785 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.409792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.411047 (+ 1255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.411091 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.412110 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.412264 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3097,"mutex_wait_us":35,"thread_start_us":2962,"threads_started":1}
I20260807 19:30:58.412652 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.402113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.8.216.129:37521)
0807 19:30:58.402995 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.403006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.403560 (+ 554us) server_negotiation.cc:407] Connection header received
0807 19:30:58.403793 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.403798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.403945 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.404130 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:58.404661 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.406168 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.408828 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.410613 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.410761 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.410993 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.412188 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.412352 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":561,"threads_started":1}
I20260807 19:30:58.414318 11400 tablet_service.cc:1467] Processing CreateTablet for tablet d5818f2f45ae4babb64e2077c6fcefab (DEFAULT_TABLE table=test-table [id=a866e0c821584589bf3d478c2605cf87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.416014 11400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5818f2f45ae4babb64e2077c6fcefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.418395 11250 tablet_service.cc:1467] Processing CreateTablet for tablet d5818f2f45ae4babb64e2077c6fcefab (DEFAULT_TABLE table=test-table [id=a866e0c821584589bf3d478c2605cf87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.418581 11100 tablet_service.cc:1467] Processing CreateTablet for tablet d5818f2f45ae4babb64e2077c6fcefab (DEFAULT_TABLE table=test-table [id=a866e0c821584589bf3d478c2605cf87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.419550 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5818f2f45ae4babb64e2077c6fcefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.419884 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5818f2f45ae4babb64e2077c6fcefab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.430411 11497 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Bootstrap starting.
I20260807 19:30:58.432786 11498 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Bootstrap starting.
I20260807 19:30:58.434644 11497 tablet_bootstrap.cc:654] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.435104 11499 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Bootstrap starting.
I20260807 19:30:58.436090 11497 log.cc:824] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:58.436475 11498 tablet_bootstrap.cc:654] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.437760 11498 log.cc:824] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:58.438580 11499 tablet_bootstrap.cc:654] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.439668 11499 log.cc:824] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Log is configured to *not* fsync() on all Append() calls
I20260807 19:30:58.439947 11497 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: No bootstrap required, opened a new log
I20260807 19:30:58.440265 11497 ts_tablet_manager.cc:1397] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:30:58.440883 11498 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: No bootstrap required, opened a new log
I20260807 19:30:58.441143 11498 ts_tablet_manager.cc:1397] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 19:30:58.442201 11499 tablet_bootstrap.cc:492] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: No bootstrap required, opened a new log
I20260807 19:30:58.442502 11499 ts_tablet_manager.cc:1397] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:30:58.449949 11499 raft_consensus.cc:348] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.450368 11499 raft_consensus.cc:374] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.450482 11499 raft_consensus.cc:729] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.451122 11499 consensus_queue.cc:260] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.451953 11498 raft_consensus.cc:348] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.452446 11498 raft_consensus.cc:374] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.452562 11498 raft_consensus.cc:729] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.453110 11499 ts_tablet_manager.cc:1428] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260807 19:30:58.453243 11498 consensus_queue.cc:260] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.451956 11497 raft_consensus.cc:348] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.454181 11497 raft_consensus.cc:374] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.454325 11497 raft_consensus.cc:729] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa460ca1ce494177b19e27c8b118dffd, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.455096 11497 consensus_queue.cc:260] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.456530 11322 heartbeater.cc:503] Master 127.8.216.190:34693 was elected leader, sending a full tablet report...
I20260807 19:30:58.457111 11475 heartbeater.cc:503] Master 127.8.216.190:34693 was elected leader, sending a full tablet report...
I20260807 19:30:58.458820 11497 ts_tablet_manager.cc:1428] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260807 19:30:58.460579 11498 ts_tablet_manager.cc:1428] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Time spent starting tablet: real 0.019s user 0.007s sys 0.010s
W20260807 19:30:58.491030 11179 tablet.cc:2369] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:30:58.492643 11506 raft_consensus.cc:482] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:58.492897 11506 raft_consensus.cc:504] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.494586 11506 leader_election.cc:288] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945), fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579)
I20260807 19:30:58.502628 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.494887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:37583 (local address 127.8.216.131:43945)
0807 19:30:58.495135 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.495144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.495848 (+ 704us) server_negotiation.cc:407] Connection header received
0807 19:30:58.496110 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.496115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.496234 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.496332 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:58.497019 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.497951 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.500987 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.501760 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.501911 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.501967 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.502228 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.502358 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:30:58.502835 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.494809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:43945 (local address 127.8.216.129:37583)
0807 19:30:58.495726 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.495748 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.495938 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.496348 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.496352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.496387 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:58.496863 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.496877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.498077 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.498085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.500813 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.500828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.501942 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.501947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.502058 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.502084 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.502195 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.502329 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":139,"thread_start_us":593,"threads_started":1}
I20260807 19:30:58.503769 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5818f2f45ae4babb64e2077c6fcefab" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c" is_pre_election: true
I20260807 19:30:58.503923 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.495170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60103 (local address 127.8.216.130:40579)
0807 19:30:58.495430 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.495438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.496143 (+ 705us) server_negotiation.cc:407] Connection header received
0807 19:30:58.496352 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.496356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.496418 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.496518 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:30:58.497175 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.498200 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.502044 (+ 3844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.502898 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.503052 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.503134 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.503522 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.503654 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:30:58.503991 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.495048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:40579 (local address 127.8.216.129:60103)
0807 19:30:58.496003 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.496018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.496200 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.496538 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.496543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.496568 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:30:58.497017 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.497034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.498353 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.498361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.500813 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.500828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.503218 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.503224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.503349 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.503373 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.503483 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.503614 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":86,"thread_start_us":590,"threads_started":1}
I20260807 19:30:58.504382 11421 raft_consensus.cc:2454] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:58.504940 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5818f2f45ae4babb64e2077c6fcefab" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd" is_pre_election: true
I20260807 19:30:58.505079 11047 leader_election.cc:302] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02; no voters:
I20260807 19:30:58.505437 11266 raft_consensus.cc:2454] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:58.505533 11506 raft_consensus.cc:2781] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:58.505678 11506 raft_consensus.cc:482] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:58.505805 11506 raft_consensus.cc:3037] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.508466 11506 raft_consensus.cc:504] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.509052 11506 leader_election.cc:288] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Requested vote from peers 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945), fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579)
I20260807 19:30:58.509485 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5818f2f45ae4babb64e2077c6fcefab" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c"
I20260807 19:30:58.509536 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5818f2f45ae4babb64e2077c6fcefab" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd"
I20260807 19:30:58.509706 11421 raft_consensus.cc:3037] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.509748 11266 raft_consensus.cc:3037] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.512179 11266 raft_consensus.cc:2454] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:58.512341 11421 raft_consensus.cc:2454] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:58.512650 11044 leader_election.cc:302] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9b241d2bd774855a90767d7c799ee02, fa460ca1ce494177b19e27c8b118dffd; no voters:
I20260807 19:30:58.513007 11506 raft_consensus.cc:2781] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:58.513470 11506 raft_consensus.cc:686] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Becoming Leader. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Running, Role: LEADER
I20260807 19:30:58.513967 11506 consensus_queue.cc:237] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } }
I20260807 19:30:58.519150 10951 catalog_manager.cc:5495] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b241d2bd774855a90767d7c799ee02 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } health_report { overall_health: HEALTHY } } }
W20260807 19:30:58.532747 11323 tablet.cc:2369] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:30:58.566258 11477 tablet.cc:2369] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:30:58 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131056, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:30:58.604068 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.591039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34693 (local address 127.0.0.1:53072)
0807 19:30:58.591708 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.591725 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.591866 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.592238 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.592242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.592958 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:30:58.593252 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.593262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.594270 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.594274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.595231 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.595242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.595938 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.595942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.596063 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.596797 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:30:58.596820 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:30:58.599083 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:30:58.601829 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.601840 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.601847 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.602259 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.602604 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:30:58.602607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:30:58.602609 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:30:58.602734 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:30:58.603232 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:30:58.603239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:30:58.603389 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.603521 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.603729 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":383,"threads_started":1}
I20260807 19:30:58.604087 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.591004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.8.216.190:34693)
0807 19:30:58.591222 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.591233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.591820 (+ 587us) server_negotiation.cc:407] Connection header received
0807 19:30:58.591976 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.591980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.592056 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.592205 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:30:58.593413 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.594150 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.595357 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.595777 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.595868 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.599266 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:30:58.599271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:30:58.599274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:30:58.599303 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:30:58.601704 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.602360 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.602364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.602366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.602506 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:30:58.602824 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:30:58.602827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:30:58.602829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:30:58.603123 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:30:58.603234 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.603615 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.603858 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":79}
I20260807 19:30:58.609895 10951 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:53072:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:30:58.611164 10951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:58.618209 11250 tablet_service.cc:1467] Processing CreateTablet for tablet 9543754b6b1340a08f8bfd370aaae6fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf958bd98e4044b8a4bca920ce5564b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:58.618032 11100 tablet_service.cc:1467] Processing CreateTablet for tablet 9543754b6b1340a08f8bfd370aaae6fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf958bd98e4044b8a4bca920ce5564b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:58.619047 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9543754b6b1340a08f8bfd370aaae6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.619081 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9543754b6b1340a08f8bfd370aaae6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.618604 11400 tablet_service.cc:1467] Processing CreateTablet for tablet 9543754b6b1340a08f8bfd370aaae6fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cf958bd98e4044b8a4bca920ce5564b4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:30:58.619427 11400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9543754b6b1340a08f8bfd370aaae6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.624498 11497 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: Bootstrap starting.
I20260807 19:30:58.625931 11497 tablet_bootstrap.cc:654] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.626426 11499 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: Bootstrap starting.
I20260807 19:30:58.627782 11499 tablet_bootstrap.cc:654] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.628469 11498 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: Bootstrap starting.
I20260807 19:30:58.630146 11497 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: No bootstrap required, opened a new log
I20260807 19:30:58.630203 11499 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: No bootstrap required, opened a new log
I20260807 19:30:58.630225 11498 tablet_bootstrap.cc:654] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.630317 11497 ts_tablet_manager.cc:1397] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 19:30:58.630379 11499 ts_tablet_manager.cc:1397] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260807 19:30:58.630977 11497 raft_consensus.cc:348] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.631251 11497 raft_consensus.cc:374] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.631364 11497 raft_consensus.cc:729] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa460ca1ce494177b19e27c8b118dffd, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.631618 11497 consensus_queue.cc:260] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.631917 11499 raft_consensus.cc:348] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.632258 11499 raft_consensus.cc:374] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.632371 11497 ts_tablet_manager.cc:1428] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:30:58.632203 11510 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.632622 11510 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.632513 11499 raft_consensus.cc:729] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.632911 11499 consensus_queue.cc:260] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.633482 11498 tablet_bootstrap.cc:492] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: No bootstrap required, opened a new log
I20260807 19:30:58.633419 11506 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.633656 11498 ts_tablet_manager.cc:1397] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260807 19:30:58.633679 11506 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.634622 11499 ts_tablet_manager.cc:1428] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260807 19:30:58.634464 11498 raft_consensus.cc:348] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.634716 11498 raft_consensus.cc:374] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.634853 11498 raft_consensus.cc:729] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.635144 11498 consensus_queue.cc:260] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.635835 11498 ts_tablet_manager.cc:1428] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 19:30:58.635659 11509 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.636330 11509 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.687611 11506 raft_consensus.cc:482] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:58.687826 11506 raft_consensus.cc:504] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.688529 11506 leader_election.cc:288] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:30:58.689101 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9543754b6b1340a08f8bfd370aaae6fc" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd" is_pre_election: true
I20260807 19:30:58.689157 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9543754b6b1340a08f8bfd370aaae6fc" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c" is_pre_election: true
I20260807 19:30:58.689368 11266 raft_consensus.cc:2454] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:58.689530 11421 raft_consensus.cc:2454] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:58.689916 11044 leader_election.cc:302] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9b241d2bd774855a90767d7c799ee02, fa460ca1ce494177b19e27c8b118dffd; no voters:
I20260807 19:30:58.690227 11506 raft_consensus.cc:2781] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:58.690320 11506 raft_consensus.cc:482] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:58.690478 11506 raft_consensus.cc:3037] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.692894 11506 raft_consensus.cc:504] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.693463 11506 leader_election.cc:288] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Requested vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:30:58.694025 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9543754b6b1340a08f8bfd370aaae6fc" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd"
I20260807 19:30:58.694023 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9543754b6b1340a08f8bfd370aaae6fc" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c"
I20260807 19:30:58.694272 11266 raft_consensus.cc:3037] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.694272 11421 raft_consensus.cc:3037] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:58.697348 11421 raft_consensus.cc:2454] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:58.697348 11266 raft_consensus.cc:2454] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:58.697818 11047 leader_election.cc:302] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02, fa460ca1ce494177b19e27c8b118dffd; no voters:
I20260807 19:30:58.698158 11506 raft_consensus.cc:2781] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:58.698370 11506 raft_consensus.cc:686] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Becoming Leader. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Running, Role: LEADER
I20260807 19:30:58.698621 11506 consensus_queue.cc:237] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.700428 11513 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: TxnStatusTablet state changed. Reason: New leader d9b241d2bd774855a90767d7c799ee02. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.700795 11513 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:58.701522 11526 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:30:58.701797 11526 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:30:58.702040 10950 catalog_manager.cc:5495] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b241d2bd774855a90767d7c799ee02 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } health_report { overall_health: UNKNOWN } } }
I20260807 19:30:58.724507 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.717726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.8.216.129:37521)
0807 19:30:58.718083 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.718087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.718111 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:30:58.718442 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.718446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.718554 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.718697 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:58.719167 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.720007 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.722069 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.722532 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.722640 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.723274 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:58.723836 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:58.723972 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.724209 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.724284 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260807 19:30:58.724632 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.717627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.0.0.1:40164)
0807 19:30:58.717932 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.717960 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.718134 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.718757 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.718763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.718786 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:30:58.719010 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.719023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.720208 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.720214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.721888 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.721907 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.722713 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.722720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.722955 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.723050 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:30:58.723963 (+ 913us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:30:58.723992 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.724105 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.724305 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 19:30:58.732841 11266 raft_consensus.cc:1261] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:58.732859 11421 raft_consensus.cc:1261] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:58.733872 11513 consensus_queue.cc:1035] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:58.734241 11506 consensus_queue.cc:1035] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:58.742826 11509 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: TxnStatusTablet state changed. Reason: New leader d9b241d2bd774855a90767d7c799ee02. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.743125 11509 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.747712 11529 mvcc.cc:205] Tried to move back new op lower bound from 7315992816558637056 to 7315992816431992832. Current Snapshot: MvccSnapshot[applied={T|T < 7315992816558637056}]
I20260807 19:30:58.748458 11506 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.748793 11506 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:58.752797 11509 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.753047 11509 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P 31ba61a198af4ef08cd63f337ee0937c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.758162 11510 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: TxnStatusTablet state changed. Reason: New leader d9b241d2bd774855a90767d7c799ee02. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.758458 11510 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.760165 11513 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.760471 11513 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P d9b241d2bd774855a90767d7c799ee02: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:30:58.766342 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.759039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.8.216.129:37521)
0807 19:30:58.759245 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.759251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.759268 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:30:58.759347 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.759352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.759448 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.759551 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:58.760111 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.761338 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.763281 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.763888 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.764031 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.764682 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:58.764873 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:58.765001 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.765744 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.765866 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:30:58.773610 11510 tablet_replica.cc:439] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } } }
I20260807 19:30:58.773898 11510 tablet_replica.cc:442] T 9543754b6b1340a08f8bfd370aaae6fc P fa460ca1ce494177b19e27c8b118dffd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:30:58.790849 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.783053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:35807 (local address 127.8.216.129:37521)
0807 19:30:58.783248 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:30:58.783256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:30:58.783976 (+ 720us) server_negotiation.cc:407] Connection header received
0807 19:30:58.784205 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:30:58.784210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:30:58.784300 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:30:58.784414 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:30:58.785113 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.786321 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.788480 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.789239 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.789372 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.789945 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:30:58.790111 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:30:58.790216 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:30:58.790432 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 19:30:58.790520 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:30:58.791384 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:30:58.783083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.8.216.129:35807)
0807 19:30:58.783850 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 19:30:58.783870 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:30:58.784031 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:30:58.784450 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:30:58.784455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:30:58.784467 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:30:58.784920 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.784934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.786482 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.786487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.788296 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:30:58.788312 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:30:58.789426 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:30:58.789433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:30:58.789625 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:30:58.789698 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:30:58.790228 (+ 530us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:30:58.790239 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:30:58.790391 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 19:30:58.790980 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":394,"threads_started":1}
I20260807 19:30:58.802937 11266 raft_consensus.cc:1261] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:58.803045 11421 raft_consensus.cc:1261] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:58.803620 11513 consensus_queue.cc:1035] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:58.803990 11537 consensus_queue.cc:1035] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:58.810896 11531 mvcc.cc:205] Tried to move back new op lower bound from 7315992816851148800 to 7315992815676973056. Current Snapshot: MvccSnapshot[applied={T|T < 7315992816851148800}]
I20260807 19:30:58.870548 10950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:58.870888 10950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:30:58.874792 10950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Sending DeleteTablet for 3 replicas of tablet d5818f2f45ae4babb64e2077c6fcefab
I20260807 19:30:58.875895 11400 tablet_service.cc:1514] Processing DeleteTablet for tablet d5818f2f45ae4babb64e2077c6fcefab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:58 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43936
I20260807 19:30:58.876099 11250 tablet_service.cc:1514] Processing DeleteTablet for tablet d5818f2f45ae4babb64e2077c6fcefab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:58 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40104
I20260807 19:30:58.876443 11100 tablet_service.cc:1514] Processing DeleteTablet for tablet d5818f2f45ae4babb64e2077c6fcefab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:30:58 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40148
I20260807 19:30:58.878068 10950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:30:58.879514 10950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:30:58.888800 11550 tablet_replica.cc:330] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: stopping tablet replica
I20260807 19:30:58.888813 11549 tablet_replica.cc:330] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: stopping tablet replica
I20260807 19:30:58.889394 11549 raft_consensus.cc:2229] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:30:58.889892 11549 raft_consensus.cc:2258] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:58.890712 11100 tablet_service.cc:1467] Processing CreateTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f (DEFAULT_TABLE table=test-table [id=990998c3c05040b281a0a8ece539a7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.890738 11399 tablet_service.cc:1467] Processing CreateTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f (DEFAULT_TABLE table=test-table [id=990998c3c05040b281a0a8ece539a7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.891261 11399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4deaa8a3f7194abbaae8944f844ffc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.891291 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4deaa8a3f7194abbaae8944f844ffc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.898379 11250 tablet_service.cc:1467] Processing CreateTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f (DEFAULT_TABLE table=test-table [id=990998c3c05040b281a0a8ece539a7af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:30:58.898963 11499 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Bootstrap starting.
I20260807 19:30:58.899102 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4deaa8a3f7194abbaae8944f844ffc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:30:58.900924 11549 ts_tablet_manager.cc:1905] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:58.901019 11499 tablet_bootstrap.cc:654] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.904605 11498 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Bootstrap starting.
I20260807 19:30:58.905189 11548 tablet_replica.cc:330] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: stopping tablet replica
I20260807 19:30:58.905627 11548 raft_consensus.cc:2229] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:30:58.906050 11548 raft_consensus.cc:2258] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:58.906869 11499 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: No bootstrap required, opened a new log
I20260807 19:30:58.907092 11498 tablet_bootstrap.cc:654] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.907119 11499 ts_tablet_manager.cc:1397] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260807 19:30:58.907967 11499 raft_consensus.cc:348] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.908298 11499 raft_consensus.cc:374] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.908402 11499 raft_consensus.cc:729] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.908746 11499 consensus_queue.cc:260] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.909345 11548 ts_tablet_manager.cc:1905] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:58.909605 11499 ts_tablet_manager.cc:1428] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:30:58.913821 11549 ts_tablet_manager.cc:1918] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:58.914070 11549 log.cc:1196] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/d5818f2f45ae4babb64e2077c6fcefab
I20260807 19:30:58.914837 11549 ts_tablet_manager.cc:1939] T d5818f2f45ae4babb64e2077c6fcefab P fa460ca1ce494177b19e27c8b118dffd: Deleting consensus metadata
I20260807 19:30:58.917160 10934 catalog_manager.cc:4841] TS fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579): tablet d5818f2f45ae4babb64e2077c6fcefab (table test-table [id=a866e0c821584589bf3d478c2605cf87]) successfully deleted
I20260807 19:30:58.917160 11550 raft_consensus.cc:2229] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:30:58.918277 11548 ts_tablet_manager.cc:1918] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:58.918438 11550 raft_consensus.cc:2258] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:30:58.918527 11548 log.cc:1196] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/d5818f2f45ae4babb64e2077c6fcefab
I20260807 19:30:58.919719 11497 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Bootstrap starting.
I20260807 19:30:58.920567 11548 ts_tablet_manager.cc:1939] T d5818f2f45ae4babb64e2077c6fcefab P 31ba61a198af4ef08cd63f337ee0937c: Deleting consensus metadata
I20260807 19:30:58.921403 11498 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: No bootstrap required, opened a new log
I20260807 19:30:58.921592 11498 ts_tablet_manager.cc:1397] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.006s
I20260807 19:30:58.921666 11497 tablet_bootstrap.cc:654] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Neither blocks nor log segments found. Creating new log.
I20260807 19:30:58.922575 10937 catalog_manager.cc:4841] TS 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945): tablet d5818f2f45ae4babb64e2077c6fcefab (table test-table [id=a866e0c821584589bf3d478c2605cf87]) successfully deleted
I20260807 19:30:58.922498 11498 raft_consensus.cc:348] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.922778 11498 raft_consensus.cc:374] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.922883 11498 raft_consensus.cc:729] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.923211 11498 consensus_queue.cc:260] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.923873 11498 ts_tablet_manager.cc:1428] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 19:30:58.924791 11497 tablet_bootstrap.cc:492] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: No bootstrap required, opened a new log
I20260807 19:30:58.924991 11497 ts_tablet_manager.cc:1397] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:30:58.925125 11550 ts_tablet_manager.cc:1905] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:30:58.925776 11497 raft_consensus.cc:348] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.926034 11497 raft_consensus.cc:374] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:30:58.926126 11497 raft_consensus.cc:729] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa460ca1ce494177b19e27c8b118dffd, State: Initialized, Role: FOLLOWER
I20260807 19:30:58.926401 11497 consensus_queue.cc:260] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:58.927030 11497 ts_tablet_manager.cc:1428] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:30:58.933768 11550 ts_tablet_manager.cc:1918] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:30:58.933954 11550 log.cc:1196] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/d5818f2f45ae4babb64e2077c6fcefab
I20260807 19:30:58.934509 11550 ts_tablet_manager.cc:1939] T d5818f2f45ae4babb64e2077c6fcefab P d9b241d2bd774855a90767d7c799ee02: Deleting consensus metadata
I20260807 19:30:58.936044 10934 catalog_manager.cc:4841] TS d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521): tablet d5818f2f45ae4babb64e2077c6fcefab (table test-table [id=a866e0c821584589bf3d478c2605cf87]) successfully deleted
I20260807 19:30:59.030475 11513 raft_consensus.cc:482] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:30:59.030696 11513 raft_consensus.cc:504] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:59.031454 11513 leader_election.cc:288] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:30:59.032045 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4deaa8a3f7194abbaae8944f844ffc3f" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd" is_pre_election: true
I20260807 19:30:59.032051 11420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4deaa8a3f7194abbaae8944f844ffc3f" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c" is_pre_election: true
I20260807 19:30:59.032351 11266 raft_consensus.cc:2454] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:59.032375 11420 raft_consensus.cc:2454] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:30:59.032850 11044 leader_election.cc:302] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9b241d2bd774855a90767d7c799ee02, fa460ca1ce494177b19e27c8b118dffd; no voters:
I20260807 19:30:59.033233 11513 raft_consensus.cc:2781] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:30:59.033370 11513 raft_consensus.cc:482] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:30:59.033499 11513 raft_consensus.cc:3037] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:59.035944 11513 raft_consensus.cc:504] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:59.036579 11513 leader_election.cc:288] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Requested vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:30:59.037146 11420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4deaa8a3f7194abbaae8944f844ffc3f" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c"
I20260807 19:30:59.037151 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4deaa8a3f7194abbaae8944f844ffc3f" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd"
I20260807 19:30:59.037359 11420 raft_consensus.cc:3037] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:59.037384 11266 raft_consensus.cc:3037] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:30:59.039397 11420 raft_consensus.cc:2454] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:59.039898 11047 leader_election.cc:302] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02; no voters:
I20260807 19:30:59.040239 11513 raft_consensus.cc:2781] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:30:59.040540 11266 raft_consensus.cc:2454] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:30:59.040421 11513 raft_consensus.cc:686] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Becoming Leader. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Running, Role: LEADER
I20260807 19:30:59.041039 11513 consensus_queue.cc:237] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:30:59.044200 10950 catalog_manager.cc:5495] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b241d2bd774855a90767d7c799ee02 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } health_report { overall_health: UNKNOWN } } }
I20260807 19:30:59.073683 11264 raft_consensus.cc:1261] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:59.074476 11537 consensus_queue.cc:1035] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:59.074611 11420 raft_consensus.cc:1261] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:30:59.075376 11537 consensus_queue.cc:1035] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:30:59.089449 11545 mvcc.cc:205] Tried to move back new op lower bound from 7315992817959301120 to 7315992817834729472. Current Snapshot: MvccSnapshot[applied={T|T < 7315992817959301120}]
W20260807 19:30:59.098564 11529 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:31:00.727037 10952 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:00.727393 10952 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:00.733484 10952 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Sending DeleteTablet for 3 replicas of tablet 4deaa8a3f7194abbaae8944f844ffc3f
I20260807 19:31:00.734388 11100 tablet_service.cc:1514] Processing DeleteTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40148
I20260807 19:31:00.734615 11250 tablet_service.cc:1514] Processing DeleteTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40104
I20260807 19:31:00.734714 11399 tablet_service.cc:1514] Processing DeleteTablet for tablet 4deaa8a3f7194abbaae8944f844ffc3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43936
I20260807 19:31:00.747753 11582 tablet_replica.cc:330] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: stopping tablet replica
I20260807 19:31:00.748231 11582 raft_consensus.cc:2229] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:00.748894 11582 raft_consensus.cc:2258] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:00.749418 11580 tablet_replica.cc:330] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: stopping tablet replica
I20260807 19:31:00.749866 11580 raft_consensus.cc:2229] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:00.750156 11581 tablet_replica.cc:330] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: stopping tablet replica
I20260807 19:31:00.750560 11581 raft_consensus.cc:2229] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:00.750561 11580 raft_consensus.cc:2258] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:00.751060 11582 ts_tablet_manager.cc:1905] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:00.751065 11581 raft_consensus.cc:2258] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:00.752192 11580 ts_tablet_manager.cc:1905] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:00.752246 11581 ts_tablet_manager.cc:1905] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:00.761591 11582 ts_tablet_manager.cc:1918] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:00.761924 11582 log.cc:1196] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/4deaa8a3f7194abbaae8944f844ffc3f
I20260807 19:31:00.761986 11580 ts_tablet_manager.cc:1918] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:00.762342 11580 log.cc:1196] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/4deaa8a3f7194abbaae8944f844ffc3f
I20260807 19:31:00.762388 11581 ts_tablet_manager.cc:1918] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:00.762701 11581 log.cc:1196] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/4deaa8a3f7194abbaae8944f844ffc3f
I20260807 19:31:00.763073 11582 ts_tablet_manager.cc:1939] T 4deaa8a3f7194abbaae8944f844ffc3f P 31ba61a198af4ef08cd63f337ee0937c: Deleting consensus metadata
I20260807 19:31:00.763460 11580 ts_tablet_manager.cc:1939] T 4deaa8a3f7194abbaae8944f844ffc3f P d9b241d2bd774855a90767d7c799ee02: Deleting consensus metadata
I20260807 19:31:00.763638 11581 ts_tablet_manager.cc:1939] T 4deaa8a3f7194abbaae8944f844ffc3f P fa460ca1ce494177b19e27c8b118dffd: Deleting consensus metadata
I20260807 19:31:00.764673 10937 catalog_manager.cc:4841] TS 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945): tablet 4deaa8a3f7194abbaae8944f844ffc3f (table test-table [id=990998c3c05040b281a0a8ece539a7af]) successfully deleted
I20260807 19:31:00.765014 10934 catalog_manager.cc:4841] TS fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579): tablet 4deaa8a3f7194abbaae8944f844ffc3f (table test-table [id=990998c3c05040b281a0a8ece539a7af]) successfully deleted
I20260807 19:31:00.765417 10934 catalog_manager.cc:4841] TS d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521): tablet 4deaa8a3f7194abbaae8944f844ffc3f (table test-table [id=990998c3c05040b281a0a8ece539a7af]) successfully deleted
Aug 07 19:31:00 dist-test-slave-8w9v krb5kdc[10901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:31:00 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:00.814497 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:00.787943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53088 (local address 127.8.216.190:34693)
0807 19:31:00.789722 (+ 1779us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:00.789727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:00.789753 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:31:00.789812 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:00.789816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:00.789880 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:00.790070 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:00.792861 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:00.793951 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:00.796657 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:00.797438 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:00.797554 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:00.805681 (+ 8127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:00.805688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:00.805691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:00.805734 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:00.810391 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:00.811534 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:00.811540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:00.811542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:00.811842 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:00.812512 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:00.812516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:00.812518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:00.812837 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:00.813207 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:00.813591 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 19:31:00.813900 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"thread_start_us":1371,"threads_started":1}
I20260807 19:31:00.819325 10952 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:00.820067 10952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:00.828378 11250 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0d6d91b5f84237947480e716ad3596 (DEFAULT_TABLE table=test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:00.828316 11100 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0d6d91b5f84237947480e716ad3596 (DEFAULT_TABLE table=test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:00.828437 11399 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0d6d91b5f84237947480e716ad3596 (DEFAULT_TABLE table=test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:00.828926 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0d6d91b5f84237947480e716ad3596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:00.828953 11100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0d6d91b5f84237947480e716ad3596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:00.829028 11399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0d6d91b5f84237947480e716ad3596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:00.837848 11591 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Bootstrap starting.
I20260807 19:31:00.840021 11592 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Bootstrap starting.
I20260807 19:31:00.841176 11593 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Bootstrap starting.
I20260807 19:31:00.842694 11591 tablet_bootstrap.cc:654] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:00.843842 11592 tablet_bootstrap.cc:654] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:00.844499 11593 tablet_bootstrap.cc:654] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:00.847589 11591 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: No bootstrap required, opened a new log
I20260807 19:31:00.847865 11591 ts_tablet_manager.cc:1397] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260807 19:31:00.848413 11592 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: No bootstrap required, opened a new log
I20260807 19:31:00.848665 11592 ts_tablet_manager.cc:1397] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260807 19:31:00.849586 11591 raft_consensus.cc:348] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.849970 11591 raft_consensus.cc:374] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:00.850193 11591 raft_consensus.cc:729] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Initialized, Role: FOLLOWER
I20260807 19:31:00.850123 11592 raft_consensus.cc:348] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.850617 11592 raft_consensus.cc:374] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:00.850871 11592 raft_consensus.cc:729] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa460ca1ce494177b19e27c8b118dffd, State: Initialized, Role: FOLLOWER
I20260807 19:31:00.850668 11591 consensus_queue.cc:260] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.851292 11592 consensus_queue.cc:260] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.852794 11591 ts_tablet_manager.cc:1428] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260807 19:31:00.855038 11592 ts_tablet_manager.cc:1428] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260807 19:31:00.856031 11593 tablet_bootstrap.cc:492] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: No bootstrap required, opened a new log
I20260807 19:31:00.856316 11593 ts_tablet_manager.cc:1397] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260807 19:31:00.857811 11593 raft_consensus.cc:348] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.858256 11593 raft_consensus.cc:374] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:00.858506 11593 raft_consensus.cc:729] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Initialized, Role: FOLLOWER
I20260807 19:31:00.858896 11593 consensus_queue.cc:260] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:00.859652 11593 ts_tablet_manager.cc:1428] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260807 19:31:01.049945 11599 raft_consensus.cc:482] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:01.050379 11599 raft_consensus.cc:504] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.052659 11599 leader_election.cc:288] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:31:01.053984 11571 raft_consensus.cc:482] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:01.054548 11571 raft_consensus.cc:504] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.055925 11571 leader_election.cc:288] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:31:01.057349 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c" is_pre_election: true
I20260807 19:31:01.057916 11421 raft_consensus.cc:2454] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:31:01.058817 11047 leader_election.cc:302] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02; no voters:
I20260807 19:31:01.058995 11264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd" is_pre_election: true
I20260807 19:31:01.059811 11571 raft_consensus.cc:2781] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:01.060066 11264 raft_consensus.cc:2454] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 0.
I20260807 19:31:01.060097 11571 raft_consensus.cc:482] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:01.060420 11571 raft_consensus.cc:3037] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.065152 11571 raft_consensus.cc:504] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.066534 11264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd"
I20260807 19:31:01.066535 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "d9b241d2bd774855a90767d7c799ee02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c"
I20260807 19:31:01.066808 11421 raft_consensus.cc:3037] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.066847 11264 raft_consensus.cc:3037] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.069116 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.052926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.8.216.130:48167)
0807 19:31:01.055126 (+ 2200us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.055159 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.055328 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.056949 (+ 1621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.056954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.057001 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:01.058453 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.058468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.061289 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.061295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.064682 (+ 3387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.064698 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.067771 (+ 3073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.067779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.068119 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.068181 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.068290 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.068442 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1958,"thread_start_us":5027,"threads_started":1}
I20260807 19:31:01.070521 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.052872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:48167 (local address 127.8.216.129:37521)
0807 19:31:01.056525 (+ 3653us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.056534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.056609 (+ 75us) server_negotiation.cc:407] Connection header received
0807 19:31:01.056680 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.056686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.056764 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.056876 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:01.058916 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.061121 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.064898 (+ 3777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.067056 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.067399 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.067495 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.069401 (+ 1906us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.069617 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3390,"thread_start_us":2882,"threads_started":1}
I20260807 19:31:01.071192 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.052944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:43945 (local address 127.8.216.130:49901)
0807 19:31:01.058727 (+ 5783us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.058750 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.058969 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.059342 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.059346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.059370 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:01.060918 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.060945 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.062966 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.062974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.066390 (+ 3416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.066408 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.069831 (+ 3423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.069842 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.070179 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.070248 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.070344 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.070548 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5557,"thread_start_us":2603,"threads_started":1}
I20260807 19:31:01.071738 11571 leader_election.cc:288] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Requested vote from peers fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579), 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945)
I20260807 19:31:01.072276 11421 raft_consensus.cc:2454] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:31:01.071698 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "fa460ca1ce494177b19e27c8b118dffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b241d2bd774855a90767d7c799ee02" is_pre_election: true
I20260807 19:31:01.072669 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.053253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:49901 (local address 127.8.216.131:43945)
0807 19:31:01.056977 (+ 3724us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.056984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.059092 (+ 2108us) server_negotiation.cc:407] Connection header received
0807 19:31:01.059163 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.059166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.059220 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.059323 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:01.061172 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.062821 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.066632 (+ 3811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.068931 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.069341 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.069424 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.070836 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.071040 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3438,"thread_start_us":3164,"threads_started":1}
I20260807 19:31:01.072903 11121 raft_consensus.cc:2379] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa460ca1ce494177b19e27c8b118dffd in current term 1: Already voted for candidate d9b241d2bd774855a90767d7c799ee02 in this term.
I20260807 19:31:01.073028 11047 leader_election.cc:302] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02; no voters:
I20260807 19:31:01.073951 11421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0d6d91b5f84237947480e716ad3596" candidate_uuid: "fa460ca1ce494177b19e27c8b118dffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31ba61a198af4ef08cd63f337ee0937c" is_pre_election: true
I20260807 19:31:01.074151 11264 raft_consensus.cc:2454] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b241d2bd774855a90767d7c799ee02 in term 1.
I20260807 19:31:01.074541 11421 raft_consensus.cc:2379] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fa460ca1ce494177b19e27c8b118dffd in current term 1: Already voted for candidate d9b241d2bd774855a90767d7c799ee02 in this term.
I20260807 19:31:01.074532 11571 raft_consensus.cc:2781] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:01.074828 11571 raft_consensus.cc:686] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Becoming Leader. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Running, Role: LEADER
I20260807 19:31:01.075299 11201 leader_election.cc:302] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa460ca1ce494177b19e27c8b118dffd; no voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02
I20260807 19:31:01.075328 11571 consensus_queue.cc:237] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.076133 11599 raft_consensus.cc:2726] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:31:01.079489 10952 catalog_manager.cc:5495] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b241d2bd774855a90767d7c799ee02 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d9b241d2bd774855a90767d7c799ee02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } health_report { overall_health: UNKNOWN } } }
I20260807 19:31:01.149537 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.145008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.8.216.129:37521)
0807 19:31:01.145196 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.145200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.145491 (+ 291us) server_negotiation.cc:407] Connection header received
0807 19:31:01.145559 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.145562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.145628 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.145712 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:01.146019 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.146721 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.148095 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.148516 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.148610 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.148988 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.149084 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.149149 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.149273 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.149333 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:31:01.158991 11421 raft_consensus.cc:1261] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:01.159017 11264 raft_consensus.cc:1261] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Refusing update from remote peer d9b241d2bd774855a90767d7c799ee02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:01.159564 11567 consensus_queue.cc:1035] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:01.159860 11571 consensus_queue.cc:1035] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:01.206568 10951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:01.207072 10951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:01.211031 10951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Sending DeleteTablet for 3 replicas of tablet 3c0d6d91b5f84237947480e716ad3596
I20260807 19:31:01.212215 11099 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0d6d91b5f84237947480e716ad3596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:01 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40148
I20260807 19:31:01.212441 11250 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0d6d91b5f84237947480e716ad3596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:01 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40104
I20260807 19:31:01.212818 11581 tablet_replica.cc:330] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: stopping tablet replica
I20260807 19:31:01.212874 11399 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0d6d91b5f84237947480e716ad3596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:01 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43936
I20260807 19:31:01.212929 11580 tablet_replica.cc:330] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: stopping tablet replica
I20260807 19:31:01.213056 11581 raft_consensus.cc:2229] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:01.213212 11582 tablet_replica.cc:330] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: stopping tablet replica
I20260807 19:31:01.213302 11581 raft_consensus.cc:2258] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:01.213333 11580 raft_consensus.cc:2229] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:01.213471 11582 raft_consensus.cc:2229] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:01.213774 11582 raft_consensus.cc:2258] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:01.213835 11580 raft_consensus.cc:2258] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:01.215159 11582 ts_tablet_manager.cc:1905] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:01.217247 11581 ts_tablet_manager.cc:1905] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:01.220932 11580 ts_tablet_manager.cc:1905] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:01.225199 11581 ts_tablet_manager.cc:1918] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:01.225514 11581 log.cc:1196] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/3c0d6d91b5f84237947480e716ad3596
I20260807 19:31:01.226251 11581 ts_tablet_manager.cc:1939] T 3c0d6d91b5f84237947480e716ad3596 P fa460ca1ce494177b19e27c8b118dffd: Deleting consensus metadata
I20260807 19:31:01.227598 10934 catalog_manager.cc:4841] TS fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579): tablet 3c0d6d91b5f84237947480e716ad3596 (table test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]) successfully deleted
I20260807 19:31:01.227396 10951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:01.228422 10951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:01.228837 11582 ts_tablet_manager.cc:1918] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:01.229141 11582 log.cc:1196] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/3c0d6d91b5f84237947480e716ad3596
I20260807 19:31:01.229359 11580 ts_tablet_manager.cc:1918] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:01.229539 11580 log.cc:1196] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/3c0d6d91b5f84237947480e716ad3596
I20260807 19:31:01.229796 11582 ts_tablet_manager.cc:1939] T 3c0d6d91b5f84237947480e716ad3596 P 31ba61a198af4ef08cd63f337ee0937c: Deleting consensus metadata
I20260807 19:31:01.230108 11580 ts_tablet_manager.cc:1939] T 3c0d6d91b5f84237947480e716ad3596 P d9b241d2bd774855a90767d7c799ee02: Deleting consensus metadata
I20260807 19:31:01.230913 10937 catalog_manager.cc:4841] TS 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945): tablet 3c0d6d91b5f84237947480e716ad3596 (table test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]) successfully deleted
I20260807 19:31:01.231122 10934 catalog_manager.cc:4841] TS d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521): tablet 3c0d6d91b5f84237947480e716ad3596 (table test-table [id=b5e77701b5984256b6c6b0cc7dcd9b0c]) successfully deleted
I20260807 19:31:01.236660 11099 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 (DEFAULT_TABLE table=test-table [id=c1de7abc88214fb09e460d66448c1076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:01.236743 11250 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 (DEFAULT_TABLE table=test-table [id=c1de7abc88214fb09e460d66448c1076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:01.237257 11250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2c04dcca864ee8bacf5112ffe51b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:01.237237 11399 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 (DEFAULT_TABLE table=test-table [id=c1de7abc88214fb09e460d66448c1076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:01.237330 11099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2c04dcca864ee8bacf5112ffe51b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:01.237650 11399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2c04dcca864ee8bacf5112ffe51b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:01.244809 11591 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Bootstrap starting.
I20260807 19:31:01.247718 11593 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Bootstrap starting.
I20260807 19:31:01.247761 11592 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Bootstrap starting.
I20260807 19:31:01.248067 11591 tablet_bootstrap.cc:654] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:01.251011 11593 tablet_bootstrap.cc:654] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:01.251005 11592 tablet_bootstrap.cc:654] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:01.255342 11591 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: No bootstrap required, opened a new log
I20260807 19:31:01.255884 11591 ts_tablet_manager.cc:1397] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260807 19:31:01.257680 11591 raft_consensus.cc:348] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.260291 11591 raft_consensus.cc:374] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:01.260601 11591 raft_consensus.cc:729] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Initialized, Role: FOLLOWER
I20260807 19:31:01.261263 11591 consensus_queue.cc:260] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.262341 11591 ts_tablet_manager.cc:1428] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 19:31:01.265205 11593 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: No bootstrap required, opened a new log
I20260807 19:31:01.265587 11593 ts_tablet_manager.cc:1397] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20260807 19:31:01.267089 11593 raft_consensus.cc:348] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.267587 11593 raft_consensus.cc:374] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:01.267812 11593 raft_consensus.cc:729] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b241d2bd774855a90767d7c799ee02, State: Initialized, Role: FOLLOWER
I20260807 19:31:01.267970 11592 tablet_bootstrap.cc:492] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: No bootstrap required, opened a new log
I20260807 19:31:01.268340 11592 ts_tablet_manager.cc:1397] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.012s
I20260807 19:31:01.268265 11593 consensus_queue.cc:260] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.269115 11593 ts_tablet_manager.cc:1428] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 19:31:01.269659 11592 raft_consensus.cc:348] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.270124 11592 raft_consensus.cc:374] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:01.270293 11592 raft_consensus.cc:729] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa460ca1ce494177b19e27c8b118dffd, State: Initialized, Role: FOLLOWER
I20260807 19:31:01.270782 11592 consensus_queue.cc:260] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.272220 11592 ts_tablet_manager.cc:1428] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:31:01.440795 11598 raft_consensus.cc:482] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:01.441265 11598 raft_consensus.cc:504] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.443674 11598 leader_election.cc:288] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521), fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579)
I20260807 19:31:01.456228 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.443828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.8.216.131:48537)
0807 19:31:01.444994 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.445016 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.445167 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.445763 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.445771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.445814 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:01.447426 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.447444 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.449597 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.449601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.452886 (+ 3285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.452899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.455393 (+ 2494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.455398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.455571 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.455598 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.455683 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.455785 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":750,"threads_started":1}
I20260807 19:31:01.456461 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.443778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48537 (local address 127.8.216.129:37521)
0807 19:31:01.443979 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.443983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.445123 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 19:31:01.445445 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.445451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.445578 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.445714 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:01.447733 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.449445 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.453109 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.455106 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.455287 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.455398 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.455754 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.455931 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:31:01.457396 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2c04dcca864ee8bacf5112ffe51b85" candidate_uuid: "31ba61a198af4ef08cd63f337ee0937c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b241d2bd774855a90767d7c799ee02" is_pre_election: true
I20260807 19:31:01.457746 11121 raft_consensus.cc:2454] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ba61a198af4ef08cd63f337ee0937c in term 0.
I20260807 19:31:01.458443 11342 leader_election.cc:302] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, d9b241d2bd774855a90767d7c799ee02; no voters:
I20260807 19:31:01.459064 11598 raft_consensus.cc:2781] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:01.459307 11598 raft_consensus.cc:482] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:01.459524 11598 raft_consensus.cc:3037] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.460069 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.445871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:47261 (local address 127.8.216.130:40579)
0807 19:31:01.447869 (+ 1998us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.447874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.448269 (+ 395us) server_negotiation.cc:407] Connection header received
0807 19:31:01.448462 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.448466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.448564 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.448648 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:01.450033 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.452283 (+ 2250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.456168 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.457975 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.458318 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.458412 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.459262 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.459454 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1825,"thread_start_us":1444,"threads_started":1}
I20260807 19:31:01.460227 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.445174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:40579 (local address 127.8.216.131:47261)
0807 19:31:01.448156 (+ 2982us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.448189 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.448333 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.448701 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.448706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.448733 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:01.449607 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.449624 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.452464 (+ 2840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.452474 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.455938 (+ 3464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.455953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.458814 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.458821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.459051 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.459076 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.459201 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.459426 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2710,"thread_start_us":2467,"threads_started":1}
I20260807 19:31:01.461630 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2c04dcca864ee8bacf5112ffe51b85" candidate_uuid: "31ba61a198af4ef08cd63f337ee0937c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd" is_pre_election: true
I20260807 19:31:01.462028 11266 raft_consensus.cc:2454] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31ba61a198af4ef08cd63f337ee0937c in term 0.
I20260807 19:31:01.463984 11598 raft_consensus.cc:504] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.464967 11598 leader_election.cc:288] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [CANDIDATE]: Term 1 election: Requested vote from peers d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521), fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579)
I20260807 19:31:01.465605 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2c04dcca864ee8bacf5112ffe51b85" candidate_uuid: "31ba61a198af4ef08cd63f337ee0937c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9b241d2bd774855a90767d7c799ee02"
I20260807 19:31:01.465605 11266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2c04dcca864ee8bacf5112ffe51b85" candidate_uuid: "31ba61a198af4ef08cd63f337ee0937c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa460ca1ce494177b19e27c8b118dffd"
I20260807 19:31:01.465932 11121 raft_consensus.cc:3037] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.466056 11266 raft_consensus.cc:3037] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:01.470970 11266 raft_consensus.cc:2454] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ba61a198af4ef08cd63f337ee0937c in term 1.
I20260807 19:31:01.471136 11121 raft_consensus.cc:2454] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31ba61a198af4ef08cd63f337ee0937c in term 1.
I20260807 19:31:01.471588 11342 leader_election.cc:302] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31ba61a198af4ef08cd63f337ee0937c, fa460ca1ce494177b19e27c8b118dffd; no voters:
I20260807 19:31:01.472019 11598 raft_consensus.cc:2781] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:01.472906 11598 raft_consensus.cc:686] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 1 LEADER]: Becoming Leader. State: Replica: 31ba61a198af4ef08cd63f337ee0937c, State: Running, Role: LEADER
I20260807 19:31:01.473562 11598 consensus_queue.cc:237] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } }
I20260807 19:31:01.478024 10952 catalog_manager.cc:5495] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c reported cstate change: term changed from 0 to 1, leader changed from <none> to 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131). New cstate: current_term: 1 leader_uuid: "31ba61a198af4ef08cd63f337ee0937c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31ba61a198af4ef08cd63f337ee0937c" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 43945 } health_report { overall_health: HEALTHY } } }
I20260807 19:31:01.505962 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.499429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.8.216.131:43945)
0807 19:31:01.499642 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.499647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.499663 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:01.499747 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.499751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.499823 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.499970 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:01.500453 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.501523 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.503389 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.503943 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.504040 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.504677 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.505283 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.505394 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.505539 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.505655 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:31:01.529166 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.520828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:39791 (local address 127.8.216.129:37521)
0807 19:31:01.521172 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.521180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.522656 (+ 1476us) server_negotiation.cc:407] Connection header received
0807 19:31:01.522820 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.522824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.522894 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.522984 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:01.523990 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.524956 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.526857 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.527661 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.527762 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.528374 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.528485 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.528555 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.528677 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.528742 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:31:01.529422 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.520971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37521 (local address 127.8.216.131:39791)
0807 19:31:01.522511 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.522553 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.522700 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.523015 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.523018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.523027 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:01.523773 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.523783 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.525083 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.525087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.526733 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.526746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.527891 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.527896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.528103 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.528221 (+ 118us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.528571 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.528579 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.528640 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.528746 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1362,"thread_start_us":996,"threads_started":1}
I20260807 19:31:01.539758 11121 raft_consensus.cc:1261] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Refusing update from remote peer 31ba61a198af4ef08cd63f337ee0937c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:01.540237 11266 raft_consensus.cc:1261] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Refusing update from remote peer 31ba61a198af4ef08cd63f337ee0937c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:01.540902 11616 consensus_queue.cc:1035] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9b241d2bd774855a90767d7c799ee02" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:31:01.541520 11598 consensus_queue.cc:1035] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa460ca1ce494177b19e27c8b118dffd" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:01.590713 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.579512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:51941 (local address 127.8.216.131:43945)
0807 19:31:01.579739 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:01.579749 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:01.581122 (+ 1373us) server_negotiation.cc:407] Connection header received
0807 19:31:01.581369 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:01.581374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:01.581465 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:01.581700 (+ 235us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:01.582697 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.584226 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.587193 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.588373 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.588546 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.589316 (+ 770us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.589684 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.589816 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:01.589988 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 19:31:01.590114 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:31:01.590888 11628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:01.578409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:43945 (local address 127.8.216.129:51941)
0807 19:31:01.580991 (+ 2582us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:01.581012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:01.581226 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:01.581733 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:01.581736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:01.581752 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:01.582508 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.582520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.584385 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.584406 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.587020 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:01.587037 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:01.588824 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:01.588831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:01.589045 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:01.589132 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:01.589838 (+ 706us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:01.589853 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 19:31:01.589945 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:31:01.590114 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2358,"thread_start_us":2036,"threads_started":1}
W20260807 19:31:01.592748 11564 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:31:03.206712 10951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:03.207010 10951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53088:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:03.213479 10951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5aab6f99998e49a78919b2b5e9b29692: Sending DeleteTablet for 3 replicas of tablet 8b2c04dcca864ee8bacf5112ffe51b85
I20260807 19:31:03.214588 11099 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:03 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40148
I20260807 19:31:03.214622 11250 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:03 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40104
I20260807 19:31:03.214761 11399 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2c04dcca864ee8bacf5112ffe51b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:03 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43936
I20260807 19:31:03.215649 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11026
I20260807 19:31:03.226432 11655 tablet_replica.cc:330] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: stopping tablet replica
I20260807 19:31:03.227101 11655 raft_consensus.cc:2229] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:03.227802 11655 raft_consensus.cc:2258] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:03.229527 11655 ts_tablet_manager.cc:1905] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:03.234171 11656 tablet_replica.cc:330] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: stopping tablet replica
I20260807 19:31:03.234560 11656 raft_consensus.cc:2229] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:03.235646 11656 raft_consensus.cc:2258] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:03.237318 11656 ts_tablet_manager.cc:1905] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:03.242159 11655 ts_tablet_manager.cc:1918] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:03.242507 11655 log.cc:1196] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/8b2c04dcca864ee8bacf5112ffe51b85
I20260807 19:31:03.243670 11655 ts_tablet_manager.cc:1939] T 8b2c04dcca864ee8bacf5112ffe51b85 P fa460ca1ce494177b19e27c8b118dffd: Deleting consensus metadata
I20260807 19:31:03.244822 11654 tablet_replica.cc:330] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: stopping tablet replica
I20260807 19:31:03.245153 11654 raft_consensus.cc:2229] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:03.245445 10934 catalog_manager.cc:4841] TS fa460ca1ce494177b19e27c8b118dffd (127.8.216.130:40579): tablet 8b2c04dcca864ee8bacf5112ffe51b85 (table test-table [id=c1de7abc88214fb09e460d66448c1076]) successfully deleted
I20260807 19:31:03.245563 11654 raft_consensus.cc:2258] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:03.247100 11654 ts_tablet_manager.cc:1905] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:03.248956 11656 ts_tablet_manager.cc:1918] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:03.249260 11656 log.cc:1196] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/8b2c04dcca864ee8bacf5112ffe51b85
I20260807 19:31:03.250089 11656 ts_tablet_manager.cc:1939] T 8b2c04dcca864ee8bacf5112ffe51b85 P 31ba61a198af4ef08cd63f337ee0937c: Deleting consensus metadata
I20260807 19:31:03.252430 10937 catalog_manager.cc:4841] TS 31ba61a198af4ef08cd63f337ee0937c (127.8.216.131:43945): tablet 8b2c04dcca864ee8bacf5112ffe51b85 (table test-table [id=c1de7abc88214fb09e460d66448c1076]) successfully deleted
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:03.257377 11654 ts_tablet_manager.cc:1918] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:31:03.257755 11654 log.cc:1196] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/8b2c04dcca864ee8bacf5112ffe51b85
I20260807 19:31:03.258744 11654 ts_tablet_manager.cc:1939] T 8b2c04dcca864ee8bacf5112ffe51b85 P d9b241d2bd774855a90767d7c799ee02: Deleting consensus metadata
I20260807 19:31:03.260443 10934 catalog_manager.cc:4841] TS d9b241d2bd774855a90767d7c799ee02 (127.8.216.129:37521): tablet 8b2c04dcca864ee8bacf5112ffe51b85 (table test-table [id=c1de7abc88214fb09e460d66448c1076]) successfully deleted
I20260807 19:31:03.269366 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:03.219962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.8.216.129:37521)
0807 19:31:03.236565 (+ 16603us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:03.236573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:03.236604 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:31:03.236669 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:03.236675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:03.236767 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:03.239377 (+ 2610us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:03.239736 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.241610 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.244465 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.245613 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.247351 (+ 1738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:03.258676 (+ 11325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:03.258717 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:03.258753 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:03.258827 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:03.264800 (+ 5973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.266049 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.266057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.266060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.266388 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.267028 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.267031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.267033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.267621 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:03.268002 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:03.268332 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:31:03.268734 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16393,"thread_start_us":16935,"threads_started":1}
W20260807 19:31:03.270505 11172 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40176
W20260807 19:31:03.271461 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11026: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:03.271772 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11026
I20260807 19:31:03.303186 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11182
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:03.319931 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:03.304069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.8.216.130:40579)
0807 19:31:03.304716 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:03.304720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:03.304737 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:31:03.304793 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:03.304796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:03.304861 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:03.304959 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:03.306502 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.307298 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.308562 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.308958 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.309057 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:03.313752 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:03.313767 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:03.313784 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:03.313820 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:03.317042 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.317922 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.317929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.317932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.318079 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.318515 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.318518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.318521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.318861 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:03.319106 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:03.319421 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:31:03.319600 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":415,"threads_started":1}
W20260807 19:31:03.320689 11317 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40114
W20260807 19:31:03.321379 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11182: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:03.321576 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11182
I20260807 19:31:03.342303 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11326
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[10901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131060, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:03.358484 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:03.343189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.8.216.131:43945)
0807 19:31:03.343840 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:03.343845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:03.343862 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:31:03.343918 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:03.343922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:03.343983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:03.344082 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:03.345452 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.346218 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.347414 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:03.347831 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:03.347923 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:03.352574 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:03.352592 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:03.352609 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:03.352648 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:03.355753 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.356583 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.356591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.356593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.356736 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:03.357075 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:03.357078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:03.357080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:03.357434 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:03.357767 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:03.357915 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 19:31:03.358094 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":407,"threads_started":1}
W20260807 19:31:03.359249 11470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43956
W20260807 19:31:03.359853 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11326: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:03.360028 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11326
I20260807 19:31:03.384877 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 10916
I20260807 19:31:03.386130 10992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:03.507817 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 10916
2026-08-07T19:31:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8770 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[11665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[11665](info): set up 2 sockets
Aug 07 19:31:03 dist-test-slave-8w9v krb5kdc[11665](info): commencing operation
krb5kdc: starting...
W20260807 19:31:04.663439 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:04 dist-test-slave-8w9v krb5kdc[11665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:04.883714 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33151
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:05.223090 11678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:05.223402 11678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:05.223598 11678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:05.231102 11678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:05.231261 11678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:05.231352 11678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:05.231459 11678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:05.231546 11678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:05.242043 11678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:05.243502 11678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:05.244900 11678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:05.257392 11686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:05.257365 11683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:05.257328 11684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:05.258338 11678 server_base.cc:1034] running on GCE node
I20260807 19:31:05.259234 11678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:05.260849 11678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:05.262125 11678 hybrid_clock.cc:648] HybridClock initialized: now 1786131065262055 us; error 60 us; skew 500 ppm
I20260807 19:31:05.262529 11678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:05 dist-test-slave-8w9v krb5kdc[11665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:05.265784 11678 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:05.267315 11678 webserver.cc:460] Webserver started at http://127.8.216.190:33767/ using document root <none> and password file <none>
I20260807 19:31:05.268056 11678 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:05.268239 11678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:05.268574 11678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:05.271639 11678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "92e480379d624e219f1c2d70c796a1c0"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "530a6546fb477ef8ec03c05ad175bb36"
server_key_iv: "6588c443eadfccaad7a71b9f113c2284"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.272550 11678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "92e480379d624e219f1c2d70c796a1c0"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "530a6546fb477ef8ec03c05ad175bb36"
server_key_iv: "6588c443eadfccaad7a71b9f113c2284"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.277933 11678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:05.281437 11696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:05.282760 11678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:05.282964 11678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "92e480379d624e219f1c2d70c796a1c0"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "530a6546fb477ef8ec03c05ad175bb36"
server_key_iv: "6588c443eadfccaad7a71b9f113c2284"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.283176 11678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:05.300312 11678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:05.310016 11678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:05.310369 11678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:05.333240 11678 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33151
I20260807 19:31:05.333251 11749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33151 every 8 connection(s)
I20260807 19:31:05.335178 11678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:05.335776 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11678
I20260807 19:31:05.336014 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:05.336931 9058 external_mini_cluster.cc:1428] Setting key 79204f6cd16d54d2c629ea70fb5f911c
I20260807 19:31:05.340909 11750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:05.355525 11750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0: Bootstrap starting.
Aug 07 19:31:05 dist-test-slave-8w9v krb5kdc[11665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:05.359419 11750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:05.360723 11750 log.cc:824] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:05.364074 11750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0: No bootstrap required, opened a new log
I20260807 19:31:05.368597 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:05.339840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.8.216.190:33151)
0807 19:31:05.340805 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:05.340825 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:05.340885 (+ 60us) server_negotiation.cc:407] Connection header received
0807 19:31:05.343874 (+ 2989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:05.343908 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:05.344528 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:05.345014 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:05.347027 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:05.349397 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:05.351670 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:05.353050 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:05.353268 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:05.359804 (+ 6536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:05.359823 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:05.359854 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:05.359932 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:05.363965 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:05.364858 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:05.364867 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:05.364876 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:05.365069 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:05.365495 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:05.365504 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:05.365507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:05.366101 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:05.366441 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:05.366969 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 19:31:05.367474 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":400,"threads_started":1}
I20260807 19:31:05.372691 11750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } }
I20260807 19:31:05.373076 11750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:05.373188 11750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e480379d624e219f1c2d70c796a1c0, State: Initialized, Role: FOLLOWER
I20260807 19:31:05.373778 11750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } }
I20260807 19:31:05.374028 11750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:05.374167 11750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:05.374361 11750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:05.376844 11750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } }
I20260807 19:31:05.377384 11750 leader_election.cc:302] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92e480379d624e219f1c2d70c796a1c0; no voters:
I20260807 19:31:05.378046 11750 leader_election.cc:288] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:05.378188 11755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:05.378901 11755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [term 1 LEADER]: Becoming Leader. State: Replica: 92e480379d624e219f1c2d70c796a1c0, State: Running, Role: LEADER
I20260807 19:31:05.379506 11755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } }
I20260807 19:31:05.380194 11750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:05.382654 11756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92e480379d624e219f1c2d70c796a1c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } } }
I20260807 19:31:05.382678 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92e480379d624e219f1c2d70c796a1c0. Latest consensus state: current_term: 1 leader_uuid: "92e480379d624e219f1c2d70c796a1c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92e480379d624e219f1c2d70c796a1c0" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33151 } } }
I20260807 19:31:05.383131 11756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:05.383312 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:05.385576 11760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:05.391651 11760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:05.401638 11760 catalog_manager.cc:1347] Generated new cluster ID: a860000d01ab4905ab9832d80bab1f98
I20260807 19:31:05.401775 11760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:05.410830 11760 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:05.412235 11760 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:05.428179 11760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92e480379d624e219f1c2d70c796a1c0: Generated new TSK 0
I20260807 19:31:05.429116 11760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:05.484726 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:05.870497 11778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:05.870908 11778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:05.871129 11778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:05.883134 11778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:05.883446 11778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:05.883690 11778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:05.898890 11778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:05.900666 11778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:05.902356 11778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:05.921260 11783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:05.921626 11784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:05.921759 11786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:05.923604 11778 server_base.cc:1034] running on GCE node
I20260807 19:31:05.924438 11778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:05.925697 11778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:05.927018 11778 hybrid_clock.cc:648] HybridClock initialized: now 1786131065926871 us; error 56 us; skew 500 ppm
I20260807 19:31:05.927469 11778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:05 dist-test-slave-8w9v krb5kdc[11665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:05.933039 11778 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:05.934897 11778 webserver.cc:460] Webserver started at http://127.8.216.129:35595/ using document root <none> and password file <none>
I20260807 19:31:05.935880 11778 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:05.936112 11778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:05.936581 11778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:05.940671 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "7afedc034315491d9558f846d116512e"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "5490e8dfacdef056f5a23109bcfa152b"
server_key_iv: "fe719ac3abcbdaac033020af7fb4b49f"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.941730 11778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "7afedc034315491d9558f846d116512e"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "5490e8dfacdef056f5a23109bcfa152b"
server_key_iv: "fe719ac3abcbdaac033020af7fb4b49f"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.948329 11778 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260807 19:31:05.953137 11794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:05.954700 11778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:31:05.954963 11778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "7afedc034315491d9558f846d116512e"
format_stamp: "Formatted at 2026-08-07 19:31:05 on dist-test-slave-8w9v"
server_key: "5490e8dfacdef056f5a23109bcfa152b"
server_key_iv: "fe719ac3abcbdaac033020af7fb4b49f"
server_key_version: "encryptionkey@0"
I20260807 19:31:05.955253 11778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:05.973126 11778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:05.979846 11778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:05.980535 11778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:05.982549 11778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:05.984727 11778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:05.984861 11778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:05.985100 11778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:05.985219 11778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:06.056243 11778 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38053
I20260807 19:31:06.056258 11914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38053 every 8 connection(s)
I20260807 19:31:06.057762 11778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:06.061054 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11778
I20260807 19:31:06.061275 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:06.061972 9058 external_mini_cluster.cc:1428] Setting key 7ebac2f586f4da7cdf881b2396d03f01
Aug 07 19:31:06 dist-test-slave-8w9v krb5kdc[11665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131065, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:06.080782 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:06.060631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:43707 (local address 127.8.216.190:33151)
0807 19:31:06.061183 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:06.061191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:06.062342 (+ 1151us) server_negotiation.cc:407] Connection header received
0807 19:31:06.064826 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:06.064834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:06.064927 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:06.065047 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:06.066523 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.067800 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.069531 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.070233 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.070385 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:06.074641 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:06.074646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:06.074649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:06.074685 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:06.077485 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:06.078294 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:06.078298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:06.078301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:06.078430 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:06.078908 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:06.078918 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:06.078919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:06.079203 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:06.079366 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:06.080107 (+ 741us) server_negotiation.cc:299] Negotiation successful
0807 19:31:06.080386 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":258,"threads_started":1}
I20260807 19:31:06.081856 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:06.061273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33151 (local address 127.8.216.129:43707)
0807 19:31:06.062163 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:06.062209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:06.064492 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:06.065249 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:06.065257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:06.065761 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:06.066331 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:06.066348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.067977 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.067981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:06.069374 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:06.069387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.070446 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.070451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:06.070655 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:06.071519 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:06.071555 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:06.074373 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:06.077655 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:06.077665 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:06.077684 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:06.078149 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:06.078550 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:06.078555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:06.078559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:06.078780 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:06.079341 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:06.079349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:06.079720 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 19:31:06.079999 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 19:31:06.080600 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":349,"threads_started":1}
I20260807 19:31:06.083726 11915 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33151
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:31:06.084079 11915 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:06.084877 11915 heartbeater.cc:511] Master 127.8.216.190:33151 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
I20260807 19:31:06.087307 11713 ts_manager.cc:194] Registered new tserver with Master: 7afedc034315491d9558f846d116512e (127.8.216.129:38053)
I20260807 19:31:06.088973 11713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:43707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:06.113689 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:06.448369 11921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:06.448676 11921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:06.448779 11921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:06.456358 11921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:06.456473 11921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:06.456598 11921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:06.466725 11921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:06.468021 11921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:06.469363 11921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:06.483201 11930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:06.483170 11928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:06.483279 11927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:06.484091 11921 server_base.cc:1034] running on GCE node
I20260807 19:31:06.484771 11921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:06.485756 11921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:06.486987 11921 hybrid_clock.cc:648] HybridClock initialized: now 1786131066486939 us; error 43 us; skew 500 ppm
I20260807 19:31:06.487341 11921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:06 dist-test-slave-8w9v krb5kdc[11665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131066, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:06.490662 11921 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:06.492308 11921 webserver.cc:460] Webserver started at http://127.8.216.130:32991/ using document root <none> and password file <none>
I20260807 19:31:06.493016 11921 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:06.493163 11921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:06.493490 11921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:06.496528 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "eebb7b5147f3482cad39aeeabdf1504b"
format_stamp: "Formatted at 2026-08-07 19:31:06 on dist-test-slave-8w9v"
server_key: "cbbdf1cc2c520bf6416e7abe49ce1eda"
server_key_iv: "4135000d98d50fbf96c22607d018a972"
server_key_version: "encryptionkey@0"
I20260807 19:31:06.497316 11921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "eebb7b5147f3482cad39aeeabdf1504b"
format_stamp: "Formatted at 2026-08-07 19:31:06 on dist-test-slave-8w9v"
server_key: "cbbdf1cc2c520bf6416e7abe49ce1eda"
server_key_iv: "4135000d98d50fbf96c22607d018a972"
server_key_version: "encryptionkey@0"
I20260807 19:31:06.502555 11921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:31:06.506075 11939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:06.507402 11921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:06.507582 11921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "eebb7b5147f3482cad39aeeabdf1504b"
format_stamp: "Formatted at 2026-08-07 19:31:06 on dist-test-slave-8w9v"
server_key: "cbbdf1cc2c520bf6416e7abe49ce1eda"
server_key_iv: "4135000d98d50fbf96c22607d018a972"
server_key_version: "encryptionkey@0"
I20260807 19:31:06.507789 11921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:06.523576 11921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:06.527705 11921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:06.528012 11921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:06.529062 11921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:06.530412 11921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:06.530520 11921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:06.530674 11921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:06.530750 11921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:06.565845 11921 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:37047
I20260807 19:31:06.565891 12056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:37047 every 8 connection(s)
I20260807 19:31:06.567150 11921 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:06.575482 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 11921
I20260807 19:31:06.575673 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:06.576231 9058 external_mini_cluster.cc:1428] Setting key e197dbe6067821dc6b44509463e434f0
Aug 07 19:31:06 dist-test-slave-8w9v krb5kdc[11665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131066, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:06.588543 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:06.569658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:36601 (local address 127.8.216.190:33151)
0807 19:31:06.569836 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:06.569841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:06.570987 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 19:31:06.572892 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:06.572898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:06.573002 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:06.573119 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:06.574647 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.575608 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.577336 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.578021 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.578185 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:06.582529 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:06.582534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:06.582537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:06.582570 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:06.585205 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:06.585985 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:06.585989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:06.585992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:06.586123 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:06.586603 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:06.586607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:06.586609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:06.586957 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:06.587117 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:06.587909 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 19:31:06.588192 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:31:06.589525 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:06.570074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33151 (local address 127.8.216.130:36601)
0807 19:31:06.570809 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:06.570848 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:06.572614 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:06.573360 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:06.573368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:06.573897 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:06.574469 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:06.574487 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.575808 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.575813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:06.577198 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:06.577211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:06.578276 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:06.578281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:06.578576 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:06.579333 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:06.579368 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:06.582294 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:06.585364 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:06.585374 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:06.585391 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:06.585833 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:06.586242 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:06.586249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:06.586255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:06.586467 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:06.587089 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:06.587097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:06.587509 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 19:31:06.587777 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:31:06.588359 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":296,"threads_started":1}
I20260807 19:31:06.591162 12057 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33151
I20260807 19:31:06.591497 12057 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:06.592316 12057 heartbeater.cc:511] Master 127.8.216.190:33151 requested a full tablet report, sending...
I20260807 19:31:06.593887 11713 ts_manager.cc:194] Registered new tserver with Master: eebb7b5147f3482cad39aeeabdf1504b (127.8.216.130:37047)
I20260807 19:31:06.595232 11713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:36601
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:06.625862 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:06.965677 12063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:06.965992 12063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:06.966092 12063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:06.973542 12063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:06.973659 12063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:06.973774 12063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:06.983986 12063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:06.985459 12063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:06.986874 12063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:07.001107 12071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:07.001334 12068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:07.001132 12069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:07.002005 12063 server_base.cc:1034] running on GCE node
I20260807 19:31:07.002615 12063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:07.003594 12063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:07.004801 12063 hybrid_clock.cc:648] HybridClock initialized: now 1786131067004761 us; error 41 us; skew 500 ppm
I20260807 19:31:07.005112 12063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:07 dist-test-slave-8w9v krb5kdc[11665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131067, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:07.008772 12063 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:07.010322 12063 webserver.cc:460] Webserver started at http://127.8.216.131:34049/ using document root <none> and password file <none>
I20260807 19:31:07.011027 12063 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:07.011240 12063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:07.011591 12063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:07.014731 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "cae09ab68d06436e825ee1741d18bc53"
format_stamp: "Formatted at 2026-08-07 19:31:07 on dist-test-slave-8w9v"
server_key: "0376248a700d32c83c4dccc32864679a"
server_key_iv: "48ea387fc976f4ff0dc1b457a10ba18c"
server_key_version: "encryptionkey@0"
I20260807 19:31:07.015581 12063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "cae09ab68d06436e825ee1741d18bc53"
format_stamp: "Formatted at 2026-08-07 19:31:07 on dist-test-slave-8w9v"
server_key: "0376248a700d32c83c4dccc32864679a"
server_key_iv: "48ea387fc976f4ff0dc1b457a10ba18c"
server_key_version: "encryptionkey@0"
I20260807 19:31:07.021142 12063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:31:07.024766 12079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:07.026539 12063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:31:07.026739 12063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "cae09ab68d06436e825ee1741d18bc53"
format_stamp: "Formatted at 2026-08-07 19:31:07 on dist-test-slave-8w9v"
server_key: "0376248a700d32c83c4dccc32864679a"
server_key_iv: "48ea387fc976f4ff0dc1b457a10ba18c"
server_key_version: "encryptionkey@0"
I20260807 19:31:07.026959 12063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:07.048799 12063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:07.053454 12063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:07.053759 12063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:07.054864 12063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:07.056504 12063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:07.056586 12063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:07.056715 12063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:07.056789 12063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:07.092109 11915 heartbeater.cc:503] Master 127.8.216.190:33151 was elected leader, sending a full tablet report...
I20260807 19:31:07.092209 12063 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:41163
I20260807 19:31:07.092235 12199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:41163 every 8 connection(s)
I20260807 19:31:07.093593 12063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:07.097934 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12063
I20260807 19:31:07.098107 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:07.098590 9058 external_mini_cluster.cc:1428] Setting key 295c0ea05a2718e21667e6e9024e4db0
Aug 07 19:31:07 dist-test-slave-8w9v krb5kdc[11665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131067, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:07.114151 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:07.096379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:51841 (local address 127.8.216.190:33151)
0807 19:31:07.096800 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:07.096804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:07.097836 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 19:31:07.099980 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:07.099984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:07.100078 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:07.100221 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:07.101908 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:07.102701 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:07.104238 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:07.104649 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:07.104742 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:07.108103 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:07.108107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:07.108110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:07.108186 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:07.110809 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:07.111571 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:07.111575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:07.111577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:07.111703 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:07.112114 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:07.112142 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:07.112146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:07.112498 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:07.112758 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:07.113571 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 19:31:07.113834 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":206,"threads_started":1}
I20260807 19:31:07.115339 12203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:07.096972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33151 (local address 127.8.216.131:51841)
0807 19:31:07.097675 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:07.097719 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:07.099713 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:07.100397 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:07.100405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:07.101044 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:07.101735 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:07.101762 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:07.102837 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:07.102841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:07.104071 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:07.104083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:07.104835 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:07.104840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:07.105054 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:07.105758 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:07.105779 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:07.107901 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:07.110954 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:07.110961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:07.110975 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:07.111445 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:07.111801 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:07.111804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:07.111807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:07.111997 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:07.112712 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:07.112719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:07.113114 (+ 395us) client_negotiation.cc:769] Sending connection context
0807 19:31:07.113435 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 19:31:07.113993 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":275,"threads_started":1}
I20260807 19:31:07.116760 12200 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33151
I20260807 19:31:07.117096 12200 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:07.117831 12200 heartbeater.cc:511] Master 127.8.216.190:33151 requested a full tablet report, sending...
I20260807 19:31:07.119292 11713 ts_manager.cc:194] Registered new tserver with Master: cae09ab68d06436e825ee1741d18bc53 (127.8.216.131:41163)
I20260807 19:31:07.120250 11713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:51841
I20260807 19:31:07.127209 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:31:07.163030 12202 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:07.160511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.8.216.190:33151)
0807 19:31:07.160694 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:07.160699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:07.161183 (+ 484us) server_negotiation.cc:407] Connection header received
0807 19:31:07.161317 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:07.161324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:07.161438 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:07.161554 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:07.162825 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40714: BlockingRecv error: recv got EOF from 127.0.0.1:40714 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:31:07.166558 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11778
W20260807 19:31:07.169581 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11778: Not authorized: Client connection negotiation failed: client connection to 127.8.216.129:38053: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:31:07.169826 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11778
I20260807 19:31:07.190454 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11921
W20260807 19:31:07.193755 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11921: Not authorized: Client connection negotiation failed: client connection to 127.8.216.130:37047: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:31:07.194053 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11921
I20260807 19:31:07.212507 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12063
W20260807 19:31:07.215837 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12063: Not authorized: Client connection negotiation failed: client connection to 127.8.216.131:41163: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:31:07.216045 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12063
I20260807 19:31:07.232923 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 11678
I20260807 19:31:07.234351 11745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:07.343627 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 11678
2026-08-07T19:31:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3832 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:07 dist-test-slave-8w9v krb5kdc[12220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:07 dist-test-slave-8w9v krb5kdc[12220](info): set up 2 sockets
Aug 07 19:31:07 dist-test-slave-8w9v krb5kdc[12220](info): commencing operation
krb5kdc: starting...
W20260807 19:31:08.482722 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:08 dist-test-slave-8w9v krb5kdc[12220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:08.704566 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46583
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:09.048612 12237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:09.048907 12237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:09.049000 12237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:09.056439 12237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:09.056563 12237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:09.056617 12237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:09.056668 12237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:09.056718 12237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:09.066892 12237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:09.068236 12237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:09.069542 12237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:09.081384 12245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:09.081481 12243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:09.082216 12242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:09.082190 12237 server_base.cc:1034] running on GCE node
I20260807 19:31:09.083173 12237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:09.084635 12237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:09.085867 12237 hybrid_clock.cc:648] HybridClock initialized: now 1786131069085828 us; error 42 us; skew 500 ppm
I20260807 19:31:09.086265 12237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:09 dist-test-slave-8w9v krb5kdc[12220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131069, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:09.096539 12237 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:09.098198 12237 webserver.cc:460] Webserver started at http://127.8.216.190:46455/ using document root <none> and password file <none>
I20260807 19:31:09.098965 12237 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:09.099128 12237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:09.099462 12237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:09.102741 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "a3bcf29e4f1949b7bf3060311cb1b906"
format_stamp: "Formatted at 2026-08-07 19:31:09 on dist-test-slave-8w9v"
server_key: "be2da0385e3fb97f1f00cf64248d92a2"
server_key_iv: "a05184faa6e16791863deb021cdd42b2"
server_key_version: "encryptionkey@0"
I20260807 19:31:09.103600 12237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "a3bcf29e4f1949b7bf3060311cb1b906"
format_stamp: "Formatted at 2026-08-07 19:31:09 on dist-test-slave-8w9v"
server_key: "be2da0385e3fb97f1f00cf64248d92a2"
server_key_iv: "a05184faa6e16791863deb021cdd42b2"
server_key_version: "encryptionkey@0"
I20260807 19:31:09.109153 12237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:31:09.112671 12253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:09.113987 12237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:31:09.114192 12237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "a3bcf29e4f1949b7bf3060311cb1b906"
format_stamp: "Formatted at 2026-08-07 19:31:09 on dist-test-slave-8w9v"
server_key: "be2da0385e3fb97f1f00cf64248d92a2"
server_key_iv: "a05184faa6e16791863deb021cdd42b2"
server_key_version: "encryptionkey@0"
I20260807 19:31:09.114403 12237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:09.147433 12237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:09.151648 12237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:09.151980 12237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:09.171422 12237 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:46583
I20260807 19:31:09.171461 12307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:46583 every 8 connection(s)
I20260807 19:31:09.172915 12237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:09.176684 12308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:09.177381 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12237
I20260807 19:31:09.177623 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:09.178264 9058 external_mini_cluster.cc:1428] Setting key 94078a1274159355352ae54e0ea7b888
I20260807 19:31:09.191166 12308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906: Bootstrap starting.
Aug 07 19:31:09 dist-test-slave-8w9v krb5kdc[12220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:09.195020 12308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:09.196412 12308 log.cc:824] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:09.199370 12308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906: No bootstrap required, opened a new log
I20260807 19:31:09.202111 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:09.180180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54064 (local address 127.8.216.190:46583)
0807 19:31:09.181063 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:09.181075 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:09.181123 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:31:09.182811 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:09.182833 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:09.183231 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:09.183589 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:09.185597 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:09.186889 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:09.188764 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:09.189473 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:09.189603 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:09.194470 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:09.194479 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:09.194496 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:09.194538 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:09.197748 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:09.198485 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:09.198492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:09.198498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:09.198662 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:09.199057 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:09.199061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:09.199063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:09.199650 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:09.199998 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:09.200420 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 19:31:09.201083 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":409,"threads_started":1}
I20260807 19:31:09.206202 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:31:09.208038 12308 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } }
I20260807 19:31:09.208501 12308 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:09.208617 12308 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3bcf29e4f1949b7bf3060311cb1b906, State: Initialized, Role: FOLLOWER
I20260807 19:31:09.209239 12308 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } }
I20260807 19:31:09.209494 12308 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:09.209647 12308 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:09.209818 12308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:09.212036 12308 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } }
I20260807 19:31:09.212607 12308 leader_election.cc:302] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3bcf29e4f1949b7bf3060311cb1b906; no voters:
I20260807 19:31:09.213243 12308 leader_election.cc:288] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:09.213404 12313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:09.214749 12313 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [term 1 LEADER]: Becoming Leader. State: Replica: a3bcf29e4f1949b7bf3060311cb1b906, State: Running, Role: LEADER
I20260807 19:31:09.215619 12308 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:09.215461 12313 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } }
I20260807 19:31:09.224977 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3bcf29e4f1949b7bf3060311cb1b906. Latest consensus state: current_term: 1 leader_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } } }
I20260807 19:31:09.225521 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:09.226236 12314 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3bcf29e4f1949b7bf3060311cb1b906" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46583 } } }
I20260807 19:31:09.226579 12314 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:09.229135 12327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:09.235134 12327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:09.244982 12327 catalog_manager.cc:1347] Generated new cluster ID: db9aea9dfd214d45b384155892103b1c
I20260807 19:31:09.245172 12327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:09.260254 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:09.252293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54072 (local address 127.8.216.190:46583)
0807 19:31:09.252480 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:09.252484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:09.253229 (+ 745us) server_negotiation.cc:407] Connection header received
0807 19:31:09.253463 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:09.253468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:09.253535 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:09.253645 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:09.254738 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:09.255498 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:09.256885 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:09.257337 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:09.257485 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:09.258654 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:09.258659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:09.258662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:09.259343 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:09.259416 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:09.259446 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:09.259571 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:09.259712 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 19:31:09.259822 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:31:09.271477 12327 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:09.273064 12327 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:09.281772 12327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3bcf29e4f1949b7bf3060311cb1b906: Generated new TSK 0
I20260807 19:31:09.282721 12327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:31:09.295138 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12237
I20260807 19:31:09.296748 12302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:09.391695 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12237
2026-08-07T19:31:09Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2042 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:09 dist-test-slave-8w9v krb5kdc[12345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:09 dist-test-slave-8w9v krb5kdc[12345](info): set up 2 sockets
Aug 07 19:31:09 dist-test-slave-8w9v krb5kdc[12345](info): commencing operation
krb5kdc: starting...
W20260807 19:31:10.523017 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:10 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:10.748780 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:34559
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:34559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:11.091212 12360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:11.091529 12360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:11.091713 12360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:11.099187 12360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:11.099313 12360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:11.099406 12360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:11.099507 12360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:11.099597 12360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:11.109591 12360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:34559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:11.111070 12360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:11.112416 12360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:11.124809 12365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:11.124763 12366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:11.124716 12368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:11.125725 12360 server_base.cc:1034] running on GCE node
I20260807 19:31:11.126591 12360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:11.128036 12360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:11.129257 12360 hybrid_clock.cc:648] HybridClock initialized: now 1786131071129214 us; error 41 us; skew 500 ppm
I20260807 19:31:11.129662 12360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:11 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:11.133081 12360 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:11.134778 12360 webserver.cc:460] Webserver started at http://127.8.216.190:44159/ using document root <none> and password file <none>
I20260807 19:31:11.135535 12360 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:11.135677 12360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:11.136027 12360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:11.139158 12360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "300795ff331b48ba91610c886279d107"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "bdcefa3a73b299e0a2e2be378d905d4a"
server_key_iv: "842db5bc3b9a72dc3e9df045250bd367"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.139948 12360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "300795ff331b48ba91610c886279d107"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "bdcefa3a73b299e0a2e2be378d905d4a"
server_key_iv: "842db5bc3b9a72dc3e9df045250bd367"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.145179 12360 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:31:11.148852 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:11.150193 12360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:11.150403 12360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "300795ff331b48ba91610c886279d107"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "bdcefa3a73b299e0a2e2be378d905d4a"
server_key_iv: "842db5bc3b9a72dc3e9df045250bd367"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.150624 12360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:11.168267 12360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:11.172983 12360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:11.173307 12360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:11.193184 12360 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:34559
I20260807 19:31:11.193216 12431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:34559 every 8 connection(s)
I20260807 19:31:11.194726 12360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:11.198644 12432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:11.201547 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12360
I20260807 19:31:11.201813 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:11.202479 9058 external_mini_cluster.cc:1428] Setting key 97e4d0105998b3ca88c8941da7ba7760
I20260807 19:31:11.212571 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107: Bootstrap starting.
I20260807 19:31:11.216661 12432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:11.218149 12432 log.cc:824] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:31:11 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:11.222004 12432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107: No bootstrap required, opened a new log
I20260807 19:31:11.229117 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:11.204753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.8.216.190:34559)
0807 19:31:11.205609 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:11.205627 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:11.205681 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:31:11.208030 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:11.208067 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:11.208685 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:11.209160 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:11.210536 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.212577 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.214483 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.215200 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.215349 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:11.220958 (+ 5609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:11.220966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:11.220981 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:11.221023 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:11.224437 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:11.225374 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:11.225381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:11.225388 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:11.225564 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:11.226012 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:11.226015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:11.226017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:11.226604 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:11.226889 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:11.227380 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 19:31:11.227941 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":353,"threads_started":1}
I20260807 19:31:11.232199 12432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } }
I20260807 19:31:11.232712 12432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:11.232823 12432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300795ff331b48ba91610c886279d107, State: Initialized, Role: FOLLOWER
I20260807 19:31:11.233453 12432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } }
I20260807 19:31:11.233695 12432 raft_consensus.cc:388] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:11.233850 12432 raft_consensus.cc:482] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:11.234030 12432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:11.236605 12432 raft_consensus.cc:504] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } }
I20260807 19:31:11.237090 12432 leader_election.cc:302] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 300795ff331b48ba91610c886279d107; no voters:
I20260807 19:31:11.237661 12432 leader_election.cc:288] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:11.237831 12437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:11.239143 12437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [term 1 LEADER]: Becoming Leader. State: Replica: 300795ff331b48ba91610c886279d107, State: Running, Role: LEADER
I20260807 19:31:11.239807 12437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } }
I20260807 19:31:11.240164 12432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:11.243131 12439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 300795ff331b48ba91610c886279d107. Latest consensus state: current_term: 1 leader_uuid: "300795ff331b48ba91610c886279d107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } } }
I20260807 19:31:11.243623 12439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:11.243391 12438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "300795ff331b48ba91610c886279d107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300795ff331b48ba91610c886279d107" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 34559 } } }
I20260807 19:31:11.243783 12438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:11.245473 12446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:11.251418 12446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:11.262423 12446 catalog_manager.cc:1347] Generated new cluster ID: 6cc19ff5482a4c1a887af4766313853b
I20260807 19:31:11.262567 12446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:11.277360 12446 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:11.279426 12446 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:11.295660 12446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 300795ff331b48ba91610c886279d107: Generated new TSK 0
I20260807 19:31:11.296962 12446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:11.356009 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:11.700511 12461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:11.700812 12461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:11.700984 12461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:11.708434 12461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:11.708586 12461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:11.708781 12461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:11.718837 12461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:11.720214 12461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:11.721573 12461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:11.735303 12467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:11.735338 12468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:11.735368 12470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:11.736439 12461 server_base.cc:1034] running on GCE node
I20260807 19:31:11.737023 12461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:11.737924 12461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:11.739164 12461 hybrid_clock.cc:648] HybridClock initialized: now 1786131071739118 us; error 33 us; skew 500 ppm
I20260807 19:31:11.739501 12461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:11 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:11.743024 12461 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:11.744904 12461 webserver.cc:460] Webserver started at http://127.8.216.129:37593/ using document root <none> and password file <none>
I20260807 19:31:11.746079 12461 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:11.746243 12461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:11.746577 12461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:11.750389 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "3c045e7d377241bc89834276f73e1a31"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "5175939e8c3fcdb5f0770f7d4138e161"
server_key_iv: "917f62489b801463db26850d84bcebe1"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.751386 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "3c045e7d377241bc89834276f73e1a31"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "5175939e8c3fcdb5f0770f7d4138e161"
server_key_iv: "917f62489b801463db26850d84bcebe1"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.759025 12461 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 19:31:11.762689 12478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:11.764108 12461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:11.764362 12461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "3c045e7d377241bc89834276f73e1a31"
format_stamp: "Formatted at 2026-08-07 19:31:11 on dist-test-slave-8w9v"
server_key: "5175939e8c3fcdb5f0770f7d4138e161"
server_key_iv: "917f62489b801463db26850d84bcebe1"
server_key_version: "encryptionkey@0"
I20260807 19:31:11.764549 12461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:11.789880 12461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:11.794965 12461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:11.795325 12461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:11.796499 12461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:11.797920 12461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:11.798007 12461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:11.798096 12461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:11.798142 12461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:11.835364 12461 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37227
I20260807 19:31:11.835417 12597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37227 every 8 connection(s)
I20260807 19:31:11.836869 12461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:11.846746 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12461
I20260807 19:31:11.846993 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:11.847851 9058 external_mini_cluster.cc:1428] Setting key 7b5fb9b4a615e79fda5d25576b12cb4b
Aug 07 19:31:11 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131071, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:11.859179 12600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:11.839457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44599 (local address 127.8.216.190:34559)
0807 19:31:11.840011 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:11.840015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:11.840930 (+ 915us) server_negotiation.cc:407] Connection header received
0807 19:31:11.842922 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:11.842928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:11.843022 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:11.843160 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:11.844791 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.845535 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.847212 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.847917 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.848098 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:11.852038 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:11.852048 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:11.852055 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:11.852102 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:11.855837 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:11.856708 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:11.856717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:11.856723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:11.856878 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:11.857301 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:11.857308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:11.857312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:11.857619 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:11.857737 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:11.858735 (+ 998us) server_negotiation.cc:299] Negotiation successful
0807 19:31:11.858912 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":266,"threads_started":1}
I20260807 19:31:11.860780 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:11.839918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34559 (local address 127.8.216.129:44599)
0807 19:31:11.840765 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:11.840804 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:11.842634 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:11.843404 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:11.843411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:11.843911 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:11.844618 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:11.844638 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.845719 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.845724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:11.847026 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:11.847040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:11.848211 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:11.848217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:11.848413 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:11.849258 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:11.849286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:11.851778 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:11.855999 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:11.856006 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:11.856021 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:11.856569 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:11.856989 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:11.856993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:11.856995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:11.857174 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:11.857764 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:11.857774 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:11.858267 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 19:31:11.858630 (+ 363us) client_negotiation.cc:240] Negotiation successful
0807 19:31:11.859244 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":374,"threads_started":1}
I20260807 19:31:11.862390 12598 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34559
I20260807 19:31:11.862722 12598 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:11.863471 12598 heartbeater.cc:511] Master 127.8.216.190:34559 requested a full tablet report, sending...
I20260807 19:31:11.865648 12394 ts_manager.cc:194] Registered new tserver with Master: 3c045e7d377241bc89834276f73e1a31 (127.8.216.129:37227)
I20260807 19:31:11.867117 12394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:44599
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:11.900451 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:12.235486 12605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:12.235800 12605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:12.235895 12605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:12.243494 12605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:12.243615 12605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:12.243729 12605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:12.254317 12605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:12.255741 12605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:12.257061 12605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:12.270579 12610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:12.270758 12613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:12.270752 12611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:12.271369 12605 server_base.cc:1034] running on GCE node
I20260807 19:31:12.271988 12605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:12.272964 12605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:12.274212 12605 hybrid_clock.cc:648] HybridClock initialized: now 1786131072274162 us; error 61 us; skew 500 ppm
I20260807 19:31:12.274544 12605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:12.277768 12605 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:12.279227 12605 webserver.cc:460] Webserver started at http://127.8.216.130:45903/ using document root <none> and password file <none>
I20260807 19:31:12.279920 12605 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:12.280079 12605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:12.280421 12605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:12.283397 12605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "6f2625d72d544e458cf5b9c590954fbe"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "a1a7930e21f34096fe0cdae92d3eb5ea"
server_key_iv: "24d196f2d10ea530cde45ce49f899f2a"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.284276 12605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "6f2625d72d544e458cf5b9c590954fbe"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "a1a7930e21f34096fe0cdae92d3eb5ea"
server_key_iv: "24d196f2d10ea530cde45ce49f899f2a"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.289740 12605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:31:12.293097 12620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.294385 12605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:31:12.294569 12605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "6f2625d72d544e458cf5b9c590954fbe"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "a1a7930e21f34096fe0cdae92d3eb5ea"
server_key_iv: "24d196f2d10ea530cde45ce49f899f2a"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.294767 12605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:12.316326 12605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:12.320305 12605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:12.320611 12605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:12.321753 12605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:12.323329 12605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:12.323402 12605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.323578 12605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:12.323629 12605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.357915 12605 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:41401
I20260807 19:31:12.357957 12737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:41401 every 8 connection(s)
I20260807 19:31:12.359277 12605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:12.363503 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12605
I20260807 19:31:12.363718 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:12.364282 9058 external_mini_cluster.cc:1428] Setting key 8b8db9240bd96abcd426f0c307149fc0
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:12.380069 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.362123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:40969 (local address 127.8.216.190:34559)
0807 19:31:12.362566 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:12.362571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:12.363563 (+ 992us) server_negotiation.cc:407] Connection header received
0807 19:31:12.365679 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:12.365684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:12.365778 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:12.365899 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:12.367454 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.368230 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.369853 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.370269 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.370402 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.374017 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:12.374023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:12.374026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:12.374061 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:12.376881 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.377649 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.377657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.377661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.377799 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.378237 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.378242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.378246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.378536 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:12.378700 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:12.379443 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 19:31:12.379693 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":212,"threads_started":1}
I20260807 19:31:12.381095 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.362694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34559 (local address 127.8.216.130:40969)
0807 19:31:12.363389 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:12.363430 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:12.365404 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:12.366156 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:12.366165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:12.366671 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:12.367250 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:12.367288 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.368394 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.368398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:12.369699 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:12.369712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.370519 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.370526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:12.370753 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.371471 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:12.371498 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:12.373766 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:12.377040 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:12.377049 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:12.377068 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:12.377519 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:12.377918 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:12.377931 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:12.377935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:12.378120 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:12.378669 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:12.378677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:12.379055 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 19:31:12.379349 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 19:31:12.379908 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":286,"threads_started":1}
I20260807 19:31:12.382555 12738 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34559
I20260807 19:31:12.382875 12738 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:12.383662 12738 heartbeater.cc:511] Master 127.8.216.190:34559 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
I20260807 19:31:12.385201 12394 ts_manager.cc:194] Registered new tserver with Master: 6f2625d72d544e458cf5b9c590954fbe (127.8.216.130:41401)
I20260807 19:31:12.386090 12394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:40969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:12.413092 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:12.749326 12744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:12.749626 12744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:12.749722 12744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:12.757124 12744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:12.757242 12744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:12.757361 12744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:12.767964 12744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:34559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:12.769446 12744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:12.770750 12744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:12.784543 12751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:12.784937 12754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:12.785450 12752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:12.785727 12744 server_base.cc:1034] running on GCE node
I20260807 19:31:12.786422 12744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:12.787390 12744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:12.788622 12744 hybrid_clock.cc:648] HybridClock initialized: now 1786131072788577 us; error 40 us; skew 500 ppm
I20260807 19:31:12.788977 12744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:12.792272 12744 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:12.793823 12744 webserver.cc:460] Webserver started at http://127.8.216.131:33479/ using document root <none> and password file <none>
I20260807 19:31:12.794601 12744 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:12.794706 12744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:12.794970 12744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:12.798043 12744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "082f712c95a64ae0ab3ac98523782a8f"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "8bc0e6aa04f106e81ea42ba04425cbb9"
server_key_iv: "da67ee6bf436f5b15f4cc5a5b01923ce"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.798765 12744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "082f712c95a64ae0ab3ac98523782a8f"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "8bc0e6aa04f106e81ea42ba04425cbb9"
server_key_iv: "da67ee6bf436f5b15f4cc5a5b01923ce"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.804069 12744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:31:12.807426 12761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.808768 12744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:12.808949 12744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "082f712c95a64ae0ab3ac98523782a8f"
format_stamp: "Formatted at 2026-08-07 19:31:12 on dist-test-slave-8w9v"
server_key: "8bc0e6aa04f106e81ea42ba04425cbb9"
server_key_iv: "da67ee6bf436f5b15f4cc5a5b01923ce"
server_key_version: "encryptionkey@0"
I20260807 19:31:12.809145 12744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:12.830655 12744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:12.835213 12744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:12.835528 12744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:12.836784 12744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:12.838536 12744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:12.838644 12744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.838768 12744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:12.838835 12744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:12.870662 12598 heartbeater.cc:503] Master 127.8.216.190:34559 was elected leader, sending a full tablet report...
I20260807 19:31:12.873461 12744 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:33023
I20260807 19:31:12.873551 12879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:33023 every 8 connection(s)
I20260807 19:31:12.874724 12744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:12.875845 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12744
I20260807 19:31:12.876067 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:12.876655 9058 external_mini_cluster.cc:1428] Setting key a1eacc802edb2cc2348e018a6e0fe193
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:12.896040 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.877619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:43803 (local address 127.8.216.190:34559)
0807 19:31:12.877815 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:12.877821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:12.879150 (+ 1329us) server_negotiation.cc:407] Connection header received
0807 19:31:12.881224 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:12.881229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:12.881330 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:12.881440 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:12.882910 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.883990 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.885603 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.886024 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.886155 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.889673 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:12.889678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:12.889681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:12.889715 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:12.892932 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.893682 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.893693 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.893695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.893814 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.894263 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.894267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.894269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.894559 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:12.894736 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:12.895454 (+ 718us) server_negotiation.cc:299] Negotiation successful
0807 19:31:12.895691 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:31:12.897084 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.878201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:34559 (local address 127.8.216.131:43803)
0807 19:31:12.879009 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:12.879045 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:12.880977 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:12.881679 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:12.881686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:12.882188 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:12.882743 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:12.882761 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.884174 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.884179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:12.885454 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:12.885467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.886263 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.886270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:12.886501 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.887121 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:12.887147 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:12.889443 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:12.893075 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:12.893084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:12.893103 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:12.893549 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:12.893933 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:12.893938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:12.893942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:12.894134 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:12.894697 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:12.894706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:12.895060 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:31:12.895326 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 19:31:12.895882 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":335,"threads_started":1}
I20260807 19:31:12.898365 12880 heartbeater.cc:347] Connected to a master server at 127.8.216.190:34559
I20260807 19:31:12.898638 12880 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:12.899319 12880 heartbeater.cc:511] Master 127.8.216.190:34559 requested a full tablet report, sending...
I20260807 19:31:12.900732 12394 ts_manager.cc:194] Registered new tserver with Master: 082f712c95a64ae0ab3ac98523782a8f (127.8.216.131:33023)
I20260807 19:31:12.901585 12394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:43803
I20260807 19:31:12.904541 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:12.957827 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.942990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.8.216.190:34559)
0807 19:31:12.943253 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:12.943262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:12.943633 (+ 371us) server_negotiation.cc:407] Connection header received
0807 19:31:12.943864 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:12.943868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:12.943952 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:12.944064 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:12.945543 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.946297 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.947577 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.948012 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.948185 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.952208 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:12.952215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:12.952220 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:12.952268 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:12.954859 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.955651 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.955658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.955661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.955813 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.956274 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.956280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.956283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.956640 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:12.956932 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:12.957243 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:31:12.957474 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 19:31:12.958494 12394 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49390
I20260807 19:31:12.960431 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12461
Aug 07 19:31:12 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:12.977385 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:12.961613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.8.216.129:37227)
0807 19:31:12.962340 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:12.962347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:12.962372 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:31:12.962430 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:12.962436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:12.962641 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:12.962784 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:12.963876 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.964972 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.966198 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:12.966786 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:12.966883 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:12.971638 (+ 4755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:12.971652 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:12.971666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:12.971711 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:12.974655 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.975331 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.975337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.975339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.975469 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:12.975790 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:12.975793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:12.975795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:12.976252 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:12.976487 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:12.976739 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:31:12.976963 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":408,"threads_started":1}
W20260807 19:31:12.978940 12593 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53120
W20260807 19:31:12.980189 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12461: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:12.980432 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12461
I20260807 19:31:13.000398 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12605
Aug 07 19:31:13 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:13.017055 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:13.001422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.8.216.130:41401)
0807 19:31:13.002028 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:13.002036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:13.002063 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:31:13.002135 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:13.002141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:13.002353 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:13.002499 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:13.003689 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:13.004811 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:13.006040 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:13.006627 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:13.006723 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:13.010944 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:13.010958 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:13.010973 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:13.011010 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:13.014232 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:13.014997 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:13.015004 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:13.015007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:13.015147 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:13.015476 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:13.015479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:13.015481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:13.015828 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:13.016071 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:13.016543 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 19:31:13.016750 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":338,"threads_started":1}
W20260807 19:31:13.018447 12733 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50166
W20260807 19:31:13.019675 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12605: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:13.019877 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12605
I20260807 19:31:13.037791 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12744
Aug 07 19:31:13 dist-test-slave-8w9v krb5kdc[12345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131072, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:13.054466 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:13.038835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.8.216.131:33023)
0807 19:31:13.039504 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:13.039511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:13.039539 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:13.039599 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:13.039605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:13.039808 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:13.040009 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:13.041432 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:13.042538 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:13.043717 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:13.044473 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:13.044568 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:13.048815 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:13.048826 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:13.048840 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:13.048878 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:13.051796 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:13.052489 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:13.052496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:13.052498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:13.052627 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:13.052960 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:13.052964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:13.052965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:13.053294 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:13.053531 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:13.053791 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 19:31:13.054077 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":344,"threads_started":1}
W20260807 19:31:13.055807 12875 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52802
W20260807 19:31:13.057010 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12744: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:31:13.057189 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12744
I20260807 19:31:13.074878 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12360
I20260807 19:31:13.076213 12427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:13.183262 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12360
2026-08-07T19:31:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3797 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:13 dist-test-slave-8w9v krb5kdc[12897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:13 dist-test-slave-8w9v krb5kdc[12897](info): set up 2 sockets
Aug 07 19:31:13 dist-test-slave-8w9v krb5kdc[12897](info): commencing operation
krb5kdc: starting...
W20260807 19:31:14.331548 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:14 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:14.559823 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38005
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:38005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:14.898885 12910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:14.899281 12910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:14.899593 12910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:14.907411 12910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:14.907605 12910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:14.907740 12910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:14.907864 12910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:14.907951 12910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:14.918452 12910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:38005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:14.920079 12910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:14.921568 12910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:14.934517 12916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:14.934593 12915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:14.934531 12918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:14.935457 12910 server_base.cc:1034] running on GCE node
I20260807 19:31:14.936435 12910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:14.938531 12910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:14.939818 12910 hybrid_clock.cc:648] HybridClock initialized: now 1786131074939765 us; error 52 us; skew 500 ppm
I20260807 19:31:14.940308 12910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:14 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131074, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:14.950775 12910 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:14.952486 12910 webserver.cc:460] Webserver started at http://127.8.216.190:33939/ using document root <none> and password file <none>
I20260807 19:31:14.953197 12910 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:14.953303 12910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:14.953583 12910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:14.956760 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "77c83b9177ec41df8e84f6287591b423"
format_stamp: "Formatted at 2026-08-07 19:31:14 on dist-test-slave-8w9v"
server_key: "c7aa2df18e3f7853de96f4781a427ff4"
server_key_iv: "5397a752688437a6e1ca597513aa7a5c"
server_key_version: "encryptionkey@0"
I20260807 19:31:14.957531 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "77c83b9177ec41df8e84f6287591b423"
format_stamp: "Formatted at 2026-08-07 19:31:14 on dist-test-slave-8w9v"
server_key: "c7aa2df18e3f7853de96f4781a427ff4"
server_key_iv: "5397a752688437a6e1ca597513aa7a5c"
server_key_version: "encryptionkey@0"
I20260807 19:31:14.962925 12910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:31:14.966451 12926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:14.967788 12910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:14.967947 12910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "77c83b9177ec41df8e84f6287591b423"
format_stamp: "Formatted at 2026-08-07 19:31:14 on dist-test-slave-8w9v"
server_key: "c7aa2df18e3f7853de96f4781a427ff4"
server_key_iv: "5397a752688437a6e1ca597513aa7a5c"
server_key_version: "encryptionkey@0"
I20260807 19:31:14.968175 12910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:15.004828 12910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:15.009171 12910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:15.009449 12910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:15.029206 12910 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:38005
I20260807 19:31:15.029239 12979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:38005 every 8 connection(s)
I20260807 19:31:15.030864 12910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:15.033403 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 12910
I20260807 19:31:15.033640 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:15.034490 9058 external_mini_cluster.cc:1428] Setting key ed8007dba4155279f4bcde52306855de
I20260807 19:31:15.035579 12980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:31:15 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:15.051213 12980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: Bootstrap starting.
I20260807 19:31:15.054646 12980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:15.055747 12980 log.cc:824] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:15.058589 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:15.036733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.8.216.190:38005)
0807 19:31:15.037608 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:15.037619 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:15.037662 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:31:15.039256 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:15.039285 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:15.039685 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:15.040040 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:15.041968 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.043474 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.045232 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.045901 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.046029 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:15.050919 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:15.050928 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:15.050946 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:15.050987 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:15.054224 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:15.055038 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:15.055049 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:15.055055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:15.055223 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:15.055635 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:15.055639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:15.055641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:15.056329 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:15.056670 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:15.057128 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 19:31:15.057579 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":388,"threads_started":1}
I20260807 19:31:15.058997 12980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: No bootstrap required, opened a new log
I20260807 19:31:15.066773 12980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } }
I20260807 19:31:15.067296 12980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:15.067456 12980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c83b9177ec41df8e84f6287591b423, State: Initialized, Role: FOLLOWER
I20260807 19:31:15.068212 12980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } }
I20260807 19:31:15.068499 12980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:15.068662 12980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:15.068856 12980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:15.071079 12980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } }
I20260807 19:31:15.071638 12980 leader_election.cc:302] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77c83b9177ec41df8e84f6287591b423; no voters:
I20260807 19:31:15.072433 12980 leader_election.cc:288] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:15.072561 12986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:15.074672 12980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:15.079509 12986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [term 1 LEADER]: Becoming Leader. State: Replica: 77c83b9177ec41df8e84f6287591b423, State: Running, Role: LEADER
I20260807 19:31:15.080075 12986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } }
I20260807 19:31:15.083369 12988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77c83b9177ec41df8e84f6287591b423. Latest consensus state: current_term: 1 leader_uuid: "77c83b9177ec41df8e84f6287591b423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } } }
I20260807 19:31:15.083508 12987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77c83b9177ec41df8e84f6287591b423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c83b9177ec41df8e84f6287591b423" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38005 } } }
I20260807 19:31:15.083807 12988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:15.083896 12987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423 [sys.catalog]: This master's current role is: LEADER
W20260807 19:31:15.089392 13000 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:31:15.089540 13000 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:31:15.089718 13001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:15.091477 13001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:15.099882 13001 catalog_manager.cc:1347] Generated new cluster ID: 5e28a01b6c684fa68a4cc9bfd29d74d1
I20260807 19:31:15.100080 13001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:15.138978 13001 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:15.140947 13001 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:15.152105 13001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77c83b9177ec41df8e84f6287591b423: Generated new TSK 0
I20260807 19:31:15.153271 13001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:15.224684 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:15.606271 13008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:15.606582 13008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:15.606681 13008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:15.615267 13008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:15.615413 13008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:15.615535 13008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:15.625906 13008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:15.627274 13008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:15.628880 13008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:15.643002 13016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:15.643035 13013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:15.643028 13014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:15.643908 13008 server_base.cc:1034] running on GCE node
I20260807 19:31:15.644744 13008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:15.645701 13008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:15.646920 13008 hybrid_clock.cc:648] HybridClock initialized: now 1786131075646863 us; error 52 us; skew 500 ppm
I20260807 19:31:15.647285 13008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:15 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131075, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:15.650701 13008 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:15.652520 13008 webserver.cc:460] Webserver started at http://127.8.216.129:35497/ using document root <none> and password file <none>
I20260807 19:31:15.653239 13008 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:15.653374 13008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:15.653716 13008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:15.656812 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "73f3cbec7b0341aa86bcbf7a839dd31e"
format_stamp: "Formatted at 2026-08-07 19:31:15 on dist-test-slave-8w9v"
server_key: "be744fb6640ee21603787d3ab92bb909"
server_key_iv: "e888bf6256ae4245ad72c2ece4fa0a6d"
server_key_version: "encryptionkey@0"
I20260807 19:31:15.657642 13008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "73f3cbec7b0341aa86bcbf7a839dd31e"
format_stamp: "Formatted at 2026-08-07 19:31:15 on dist-test-slave-8w9v"
server_key: "be744fb6640ee21603787d3ab92bb909"
server_key_iv: "e888bf6256ae4245ad72c2ece4fa0a6d"
server_key_version: "encryptionkey@0"
I20260807 19:31:15.663136 13008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:31:15.666760 13023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:15.668211 13008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:15.668396 13008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "73f3cbec7b0341aa86bcbf7a839dd31e"
format_stamp: "Formatted at 2026-08-07 19:31:15 on dist-test-slave-8w9v"
server_key: "be744fb6640ee21603787d3ab92bb909"
server_key_iv: "e888bf6256ae4245ad72c2ece4fa0a6d"
server_key_version: "encryptionkey@0"
I20260807 19:31:15.668630 13008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:15.689823 13008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:15.694164 13008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:15.694494 13008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:15.695533 13008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:15.696997 13008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:15.697105 13008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:15.697266 13008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:15.697340 13008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:15.732340 13008 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:40723
I20260807 19:31:15.732391 13150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:40723 every 8 connection(s)
I20260807 19:31:15.733693 13008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:15.741677 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13008
I20260807 19:31:15.741889 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:15.742522 9058 external_mini_cluster.cc:1428] Setting key 945e659c4e24c83c2952571093019323
Aug 07 19:31:15 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131075, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:15.757684 13153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:15.736431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:43807 (local address 127.8.216.190:38005)
0807 19:31:15.736858 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:15.736863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:15.737914 (+ 1051us) server_negotiation.cc:407] Connection header received
0807 19:31:15.739811 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:15.739816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:15.739910 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:15.740045 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:15.741652 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.742493 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.744593 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.745262 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.745401 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:15.750646 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:15.750652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:15.750657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:15.750724 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:15.753879 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:15.754934 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:15.754942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:15.754948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:15.755098 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:15.755532 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:15.755539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:15.755546 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:15.755953 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:15.756165 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:15.756915 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 19:31:15.757193 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":211,"threads_started":1}
I20260807 19:31:15.758303 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:15.737031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38005 (local address 127.8.216.129:43807)
0807 19:31:15.737747 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:15.737789 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:15.739535 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:15.740325 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:15.740333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:15.740914 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:15.741477 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:15.741494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.742667 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.742677 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:15.744390 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:15.744405 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:15.745615 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:15.745626 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:15.745976 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:15.746856 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:15.746896 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:15.750338 (+ 3442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:15.754048 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:15.754060 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:15.754083 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:15.754774 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:15.755209 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:15.755212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:15.755215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:15.755406 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:15.756091 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:15.756097 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:15.756503 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 19:31:15.756740 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 19:31:15.757152 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":281,"threads_started":1}
I20260807 19:31:15.759971 13151 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38005
I20260807 19:31:15.760351 13151 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:15.761319 13151 heartbeater.cc:511] Master 127.8.216.190:38005 requested a full tablet report, sending...
I20260807 19:31:15.763792 12944 ts_manager.cc:194] Registered new tserver with Master: 73f3cbec7b0341aa86bcbf7a839dd31e (127.8.216.129:40723)
I20260807 19:31:15.765343 12944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:43807
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:15.799379 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:16.132263 13157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:16.132649 13157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:16.132858 13157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:16.140270 13157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:16.140412 13157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:16.140607 13157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:16.150629 13157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:16.151981 13157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:16.153403 13157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:16.167299 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:16.167263 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:16.167346 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:16.168092 13157 server_base.cc:1034] running on GCE node
I20260807 19:31:16.168744 13157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:16.169680 13157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:16.170889 13157 hybrid_clock.cc:648] HybridClock initialized: now 1786131076170835 us; error 36 us; skew 500 ppm
I20260807 19:31:16.171224 13157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:16.174612 13157 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:16.176246 13157 webserver.cc:460] Webserver started at http://127.8.216.130:35891/ using document root <none> and password file <none>
I20260807 19:31:16.177030 13157 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:16.177182 13157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:16.177517 13157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:16.180768 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "3373c3dcd63f403ea9f0d01e5e164bc1"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "c318ad352d092bb4d10969c0c195cb11"
server_key_iv: "3d033102451004a4a330ebf12fba5197"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.181597 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "3373c3dcd63f403ea9f0d01e5e164bc1"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "c318ad352d092bb4d10969c0c195cb11"
server_key_iv: "3d033102451004a4a330ebf12fba5197"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.186933 13157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:31:16.190673 13172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.192176 13157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:16.192363 13157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "3373c3dcd63f403ea9f0d01e5e164bc1"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "c318ad352d092bb4d10969c0c195cb11"
server_key_iv: "3d033102451004a4a330ebf12fba5197"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.192584 13157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:16.223999 13157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:16.228363 13157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:16.228675 13157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:16.229768 13157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:16.231331 13157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:16.231437 13157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.231593 13157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:16.231676 13157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.266623 13157 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:42917
I20260807 19:31:16.266669 13290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:42917 every 8 connection(s)
I20260807 19:31:16.268049 13157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:16.273043 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13157
I20260807 19:31:16.273258 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:16.273859 9058 external_mini_cluster.cc:1428] Setting key e932871f0723019efb2343eaebbfe13b
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:16.294343 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.270787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:42903 (local address 127.8.216.190:38005)
0807 19:31:16.271279 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:16.271284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:16.272314 (+ 1030us) server_negotiation.cc:407] Connection header received
0807 19:31:16.275007 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:16.275014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:16.275106 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:16.275199 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:16.277544 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.278417 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.281181 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.281668 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.281773 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.286610 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:16.286621 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:16.286629 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:16.286682 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:16.290591 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.291533 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.291541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.291548 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.291715 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.292206 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.292214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.292220 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.292642 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:16.292799 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:16.293556 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 19:31:16.293827 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":278,"threads_started":1}
I20260807 19:31:16.295082 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.271264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38005 (local address 127.8.216.130:42903)
0807 19:31:16.272057 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:16.272105 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:16.274704 (+ 2599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:16.275507 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:16.275520 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:16.276390 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:16.277313 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:16.277345 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.278606 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.278615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:16.280969 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:16.280992 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.281936 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.281955 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:16.282323 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.283181 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:16.283220 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:16.286288 (+ 3068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:16.290859 (+ 4571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:16.290867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:16.290881 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:16.291371 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:16.291834 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:16.291837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:16.291839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:16.292037 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:16.292780 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:16.292786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:16.293157 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 19:31:16.293395 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 19:31:16.293811 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":283,"threads_started":1}
I20260807 19:31:16.296752 13291 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38005
I20260807 19:31:16.297112 13291 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:16.298096 13291 heartbeater.cc:511] Master 127.8.216.190:38005 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
I20260807 19:31:16.299822 12944 ts_manager.cc:194] Registered new tserver with Master: 3373c3dcd63f403ea9f0d01e5e164bc1 (127.8.216.130:42917)
I20260807 19:31:16.300809 12944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:42903
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:16.329118 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:16.663113 13297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:16.663391 13297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:16.663486 13297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:16.670817 13297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:16.670938 13297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:16.671048 13297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:16.681339 13297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38005
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:16.682673 13297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:16.683972 13297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:16.697423 13305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:16.697543 13303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:16.697512 13302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:16.698391 13297 server_base.cc:1034] running on GCE node
I20260807 19:31:16.699039 13297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:16.699980 13297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:16.701207 13297 hybrid_clock.cc:648] HybridClock initialized: now 1786131076701122 us; error 71 us; skew 500 ppm
I20260807 19:31:16.701529 13297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:16.704684 13297 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:16.706135 13297 webserver.cc:460] Webserver started at http://127.8.216.131:37887/ using document root <none> and password file <none>
I20260807 19:31:16.706825 13297 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:16.706959 13297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:16.707275 13297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:16.710214 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "c276e9c0a590446c8e7e62fd78757b37"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "128fe6c0e90c5193f2c436b390893dfd"
server_key_iv: "9d09b630be5109b37bebc1a65730d36b"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.711028 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "c276e9c0a590446c8e7e62fd78757b37"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "128fe6c0e90c5193f2c436b390893dfd"
server_key_iv: "9d09b630be5109b37bebc1a65730d36b"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.716086 13297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:31:16.719547 13312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.720847 13297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:16.721014 13297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "c276e9c0a590446c8e7e62fd78757b37"
format_stamp: "Formatted at 2026-08-07 19:31:16 on dist-test-slave-8w9v"
server_key: "128fe6c0e90c5193f2c436b390893dfd"
server_key_iv: "9d09b630be5109b37bebc1a65730d36b"
server_key_version: "encryptionkey@0"
I20260807 19:31:16.721228 13297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:16.736214 13297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:16.740504 13297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:16.740840 13297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:16.741835 13297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:16.743191 13297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:16.743265 13297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.743438 13297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:16.743494 13297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:16.769181 13151 heartbeater.cc:503] Master 127.8.216.190:38005 was elected leader, sending a full tablet report...
I20260807 19:31:16.778384 13297 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:33939
I20260807 19:31:16.778426 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:33939 every 8 connection(s)
I20260807 19:31:16.779778 13297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:16.782378 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13297
I20260807 19:31:16.782590 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:16.783070 9058 external_mini_cluster.cc:1428] Setting key 38a5cceac3267bb9d8ee1c99baa317d7
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:16.801143 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.782640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:50253 (local address 127.8.216.190:38005)
0807 19:31:16.783711 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:16.783718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:16.784182 (+ 464us) server_negotiation.cc:407] Connection header received
0807 19:31:16.786255 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:16.786261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:16.786362 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:16.786477 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:16.788115 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.788924 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.790535 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.790950 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.791090 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.794769 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:16.794774 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:16.794777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:16.794811 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:16.797699 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.798654 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.798660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.798664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.798819 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.799256 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.799263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.799268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.799566 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:16.799751 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:16.800536 (+ 785us) server_negotiation.cc:299] Negotiation successful
0807 19:31:16.800780 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893}
I20260807 19:31:16.802116 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.783244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38005 (local address 127.8.216.131:50253)
0807 19:31:16.784006 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:16.784045 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:16.785968 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:16.786756 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:16.786770 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:16.787329 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:16.787929 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:16.787949 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.789105 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.789109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:16.790366 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:16.790379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.791186 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.791193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:16.791444 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.792217 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:16.792241 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:16.794559 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:16.797936 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:16.797946 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:16.797964 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:16.798521 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:16.798938 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:16.798943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:16.798947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:16.799139 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:16.799700 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:16.799708 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:16.800076 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 19:31:16.800425 (+ 349us) client_negotiation.cc:240] Negotiation successful
0807 19:31:16.800997 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":398,"threads_started":1}
I20260807 19:31:16.803527 13432 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38005
I20260807 19:31:16.803812 13432 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:16.804600 13432 heartbeater.cc:511] Master 127.8.216.190:38005 requested a full tablet report, sending...
I20260807 19:31:16.806035 12944 ts_manager.cc:194] Registered new tserver with Master: c276e9c0a590446c8e7e62fd78757b37 (127.8.216.131:33939)
I20260807 19:31:16.806892 12944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:50253
I20260807 19:31:16.811420 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:16.867712 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.851986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.8.216.129:40723)
0807 19:31:16.852671 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:16.852678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:16.852707 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:31:16.852921 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:16.852930 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:16.853173 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:16.853380 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:16.854872 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.855936 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.857266 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.857884 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.857991 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.861866 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:16.861880 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:16.861896 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:16.861937 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:16.865110 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.865728 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.865735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.865737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.865858 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.866181 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.866184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.866186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.866537 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:16.866771 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:16.867065 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 19:31:16.867318 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":315,"threads_started":1}
Aug 07 19:31:16 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:16.886617 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.872435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.8.216.190:38005)
0807 19:31:16.872662 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:16.872669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:16.872877 (+ 208us) server_negotiation.cc:407] Connection header received
0807 19:31:16.873117 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:16.873121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:16.873201 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:16.873286 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:16.874572 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.875445 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.876764 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.877176 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.877264 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.881339 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:16.881345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:16.881348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:16.881394 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:16.884059 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.884766 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.884770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.884772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.884900 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.885215 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.885218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.885220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.885573 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:16.885766 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:16.886025 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 19:31:16.886197 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:31:16.887220 12944 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56798
I20260807 19:31:16.888396 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13008
I20260807 19:31:16.902833 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:16.889484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.8.216.129:40723)
0807 19:31:16.889676 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:16.889680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:16.889811 (+ 131us) server_negotiation.cc:407] Connection header received
0807 19:31:16.889920 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:16.889923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:16.890023 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:16.890109 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:16.891340 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.892339 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.893585 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:16.894023 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:16.894135 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:16.896635 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:16.896642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:16.896646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:16.896710 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:16.900243 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.900952 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.900957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.900959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.901113 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:16.901470 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:16.901474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:16.901476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:16.901792 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:16.902005 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:16.902292 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 19:31:16.902503 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:31:16.903652 13146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:16.994985 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13008
I20260807 19:31:17.016424 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13157
Aug 07 19:31:17 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:17.036062 13444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:17.017443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.8.216.130:42917)
0807 19:31:17.018025 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:17.018032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:17.018059 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:31:17.018124 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:17.018130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:17.018348 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:17.018481 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:17.020306 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:17.021393 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:17.023219 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:17.023796 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:17.023962 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:17.028405 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:17.028419 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:17.028435 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:17.028475 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:17.033050 (+ 4575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:17.033912 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:17.033920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:17.033922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:17.034057 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:17.034451 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:17.034457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:17.034463 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:17.034829 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:17.035060 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:17.035423 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 19:31:17.035630 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":289,"threads_started":1}
I20260807 19:31:17.037379 13286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:17.122424 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13157
I20260807 19:31:17.142838 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13297
Aug 07 19:31:17 dist-test-slave-8w9v krb5kdc[12897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:17.159950 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:17.143911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.8.216.131:33939)
0807 19:31:17.144643 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:17.144650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:17.144681 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:31:17.144740 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:17.144747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:17.144957 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:17.145088 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:17.146407 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:17.147442 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:17.148856 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:17.149434 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:17.149531 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:17.154143 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:17.154155 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:17.154170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:17.154206 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:17.157304 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:17.157957 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:17.157963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:17.157966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:17.158081 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:17.158507 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:17.158510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:17.158512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:17.158837 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:17.159090 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:17.159351 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:31:17.159556 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":409,"threads_started":1}
I20260807 19:31:17.161260 13427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:17.245433 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13297
I20260807 19:31:17.263912 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 12910
I20260807 19:31:17.265255 12975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:17.372893 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 12910
2026-08-07T19:31:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4189 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:17 dist-test-slave-8w9v krb5kdc[13451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:17 dist-test-slave-8w9v krb5kdc[13451](info): set up 2 sockets
Aug 07 19:31:17 dist-test-slave-8w9v krb5kdc[13451](info): commencing operation
krb5kdc: starting...
W20260807 19:31:18.536203 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:18 dist-test-slave-8w9v krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:18.770037 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35515
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:35515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:19.111745 13464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:19.112114 13464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:19.112342 13464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:19.119830 13464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:19.119982 13464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:19.120098 13464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:19.120213 13464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:19.120298 13464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:19.130698 13464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:35515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:19.132274 13464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:19.133692 13464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:19.146306 13470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:19.146340 13472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:19.146406 13469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:19.147154 13464 server_base.cc:1034] running on GCE node
I20260807 19:31:19.148051 13464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:19.149614 13464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:19.150820 13464 hybrid_clock.cc:648] HybridClock initialized: now 1786131079150748 us; error 57 us; skew 500 ppm
I20260807 19:31:19.151206 13464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:19 dist-test-slave-8w9v krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131079, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:19.154390 13464 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:19.155604 13464 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:19.155745 13464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:19.156097 13464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:19.159252 13464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "ddce4ecf028046ebb01419091d98f10a"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "d9a45b313f9348a4589ee7f9441d54d2"
server_key_iv: "9a9635056ddaf4a9b68cd7006faeec5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.160113 13464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "ddce4ecf028046ebb01419091d98f10a"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "d9a45b313f9348a4589ee7f9441d54d2"
server_key_iv: "9a9635056ddaf4a9b68cd7006faeec5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.165462 13464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:19.169095 13478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:19.170341 13464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:19.170531 13464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "ddce4ecf028046ebb01419091d98f10a"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "d9a45b313f9348a4589ee7f9441d54d2"
server_key_iv: "9a9635056ddaf4a9b68cd7006faeec5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.170754 13464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:19.204986 13464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:19.209280 13464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:19.209594 13464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:19.229166 13464 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:35515
I20260807 19:31:19.229179 13534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:35515 every 8 connection(s)
I20260807 19:31:19.230201 13464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:19.233753 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13464
I20260807 19:31:19.233989 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:19.234349 13535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:19.234861 9058 external_mini_cluster.cc:1428] Setting key f38e711b15b9628e72b4cdd36e377ef8
I20260807 19:31:19.247732 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: Bootstrap starting.
Aug 07 19:31:19 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:19.251436 13535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:19.252779 13535 log.cc:824] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:19.256248 13535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: No bootstrap required, opened a new log
I20260807 19:31:19.259151 13538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:19.237144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.8.216.190:35515)
0807 19:31:19.237977 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:19.237992 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:19.238035 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:31:19.239587 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:19.239612 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:19.240012 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:19.240472 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:19.242043 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.243319 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.244652 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.245383 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.245621 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:19.250018 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:19.250026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:19.250041 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:19.250081 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:19.253842 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:19.254695 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:19.254701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:19.254708 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:19.254885 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:19.255312 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:19.255315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:19.255317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:19.255834 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:19.256081 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:19.257300 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0807 19:31:19.257752 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":364,"threads_started":1}
I20260807 19:31:19.265856 13535 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } }
I20260807 19:31:19.266237 13535 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:19.266315 13535 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddce4ecf028046ebb01419091d98f10a, State: Initialized, Role: FOLLOWER
I20260807 19:31:19.266819 13535 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } }
I20260807 19:31:19.267019 13535 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:19.267122 13535 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:19.267251 13535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:19.269579 13535 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } }
I20260807 19:31:19.270062 13535 leader_election.cc:302] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddce4ecf028046ebb01419091d98f10a; no voters:
I20260807 19:31:19.270594 13535 leader_election.cc:288] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:19.270769 13540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:19.272260 13540 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [term 1 LEADER]: Becoming Leader. State: Replica: ddce4ecf028046ebb01419091d98f10a, State: Running, Role: LEADER
I20260807 19:31:19.272800 13540 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } }
I20260807 19:31:19.272913 13535 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:19.275753 13541 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddce4ecf028046ebb01419091d98f10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } } }
I20260807 19:31:19.275954 13542 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddce4ecf028046ebb01419091d98f10a. Latest consensus state: current_term: 1 leader_uuid: "ddce4ecf028046ebb01419091d98f10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddce4ecf028046ebb01419091d98f10a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35515 } } }
I20260807 19:31:19.276373 13541 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:19.276391 13542 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:19.278218 13549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:19.285101 13549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:19.294945 13549 catalog_manager.cc:1347] Generated new cluster ID: ce02229e5add4931aecadb9d9ec87a38
I20260807 19:31:19.295082 13549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:19.305888 13549 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:19.308012 13549 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:19.322314 13549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: Generated new TSK 0
I20260807 19:31:19.323464 13549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:19.380164 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:19.726183 13562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:19.726485 13562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:19.726658 13562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:19.734229 13562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:19.734355 13562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:19.734543 13562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:19.745733 13562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:19.747678 13562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:19.749526 13562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:19.763612 13568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:19.763845 13567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:19.763798 13571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:19.764659 13562 server_base.cc:1034] running on GCE node
I20260807 19:31:19.765282 13562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:19.766240 13562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:19.767478 13562 hybrid_clock.cc:648] HybridClock initialized: now 1786131079767430 us; error 60 us; skew 500 ppm
I20260807 19:31:19.767799 13562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:19 dist-test-slave-8w9v krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131079, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:19.771036 13562 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:19.772337 13562 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:19.772485 13562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:19.772830 13562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:19.776582 13562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "fabb8ab4a6fd409897126494f47b0e04"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "1faf4d42df7e21ef9e69408ea7549b5e"
server_key_iv: "cb4249bd270f37dbe07b8d6e5bb04871"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.777690 13562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "fabb8ab4a6fd409897126494f47b0e04"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "1faf4d42df7e21ef9e69408ea7549b5e"
server_key_iv: "cb4249bd270f37dbe07b8d6e5bb04871"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.783046 13562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:31:19.786728 13579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:19.788199 13562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:19.788403 13562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "fabb8ab4a6fd409897126494f47b0e04"
format_stamp: "Formatted at 2026-08-07 19:31:19 on dist-test-slave-8w9v"
server_key: "1faf4d42df7e21ef9e69408ea7549b5e"
server_key_iv: "cb4249bd270f37dbe07b8d6e5bb04871"
server_key_version: "encryptionkey@0"
I20260807 19:31:19.788661 13562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:19.809360 13562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:19.813616 13562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:19.813925 13562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:19.814975 13562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:19.816514 13562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:19.816639 13562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:19.816780 13562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:19.816900 13562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:19.851341 13562 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:43453
I20260807 19:31:19.851401 13697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:43453 every 8 connection(s)
I20260807 19:31:19.852280 13562 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:19.855508 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13562
I20260807 19:31:19.855717 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:19.856462 9058 external_mini_cluster.cc:1428] Setting key 35856768f5540bc5b4436aa48d7eb174
Aug 07 19:31:19 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131079, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:19.874570 13701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:19.856344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:40271 (local address 127.8.216.190:35515)
0807 19:31:19.856929 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:19.856935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:19.856973 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:31:19.859132 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:19.859140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:19.859229 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:19.859354 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:19.860926 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.862094 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.863645 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.864400 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.864550 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:19.868439 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:19.868444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:19.868449 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:19.868482 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:19.871036 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:19.871805 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:19.871815 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:19.871817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:19.871948 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:19.872444 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:19.872448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:19.872450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:19.872789 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:19.872962 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:19.873895 (+ 933us) server_negotiation.cc:299] Negotiation successful
0807 19:31:19.874150 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":792,"threads_started":1}
I20260807 19:31:19.875756 13700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:19.855748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35515 (local address 127.8.216.129:40271)
0807 19:31:19.856693 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:19.856741 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:19.858849 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:19.859548 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:19.859557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:19.860095 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:19.860746 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:19.860764 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.862254 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.862258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:19.863488 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:19.863500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:19.864610 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:19.864615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:19.864803 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:19.865525 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:19.865547 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:19.868214 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:19.871187 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:19.871197 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:19.871214 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:19.871671 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:19.872068 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:19.872074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:19.872077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:19.872311 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:19.872931 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:19.872939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:19.873367 (+ 428us) client_negotiation.cc:769] Sending connection context
0807 19:31:19.873771 (+ 404us) client_negotiation.cc:240] Negotiation successful
0807 19:31:19.874366 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":634,"threads_started":1}
I20260807 19:31:19.877578 13698 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35515
I20260807 19:31:19.877904 13698 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:19.878599 13698 heartbeater.cc:511] Master 127.8.216.190:35515 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:31:19.880908 13497 ts_manager.cc:194] Registered new tserver with Master: fabb8ab4a6fd409897126494f47b0e04 (127.8.216.129:43453)
I20260807 19:31:19.882388 13497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:40271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:19.912242 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:20.244620 13704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:20.244907 13704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:20.245097 13704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:20.252636 13704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:20.252772 13704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:20.252975 13704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:20.263381 13704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:20.264870 13704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:20.266244 13704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:20.279559 13709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:20.279597 13712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:20.279631 13710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:20.280434 13704 server_base.cc:1034] running on GCE node
I20260807 19:31:20.281014 13704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:20.281919 13704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:20.283128 13704 hybrid_clock.cc:648] HybridClock initialized: now 1786131080283076 us; error 40 us; skew 500 ppm
I20260807 19:31:20.283411 13704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:20 dist-test-slave-8w9v krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131080, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:20.292529 13704 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:20.293895 13704 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:20.294024 13704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:20.294384 13704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:20.297528 13704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "03ded436a38a4a17a349ce36b40edd09"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "5f31a048067c5aabc3af228d5b99ca66"
server_key_iv: "c227a3d09d5ed0babc462bad82622646"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.298333 13704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "03ded436a38a4a17a349ce36b40edd09"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "5f31a048067c5aabc3af228d5b99ca66"
server_key_iv: "c227a3d09d5ed0babc462bad82622646"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.303524 13704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:31:20.307130 13719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.308540 13704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:31:20.308672 13704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "03ded436a38a4a17a349ce36b40edd09"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "5f31a048067c5aabc3af228d5b99ca66"
server_key_iv: "c227a3d09d5ed0babc462bad82622646"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.308853 13704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:20.333359 13704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:20.337688 13704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:20.337951 13704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:20.338872 13704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:20.340294 13704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:20.340387 13704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.340461 13704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:20.340517 13704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.375061 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:33175 every 8 connection(s)
I20260807 19:31:20.375279 13704 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:33175
I20260807 19:31:20.376325 13704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:20.386123 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13704
I20260807 19:31:20.386380 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:20.386891 9058 external_mini_cluster.cc:1428] Setting key 751b8a622c567081e98508a771b3e04c
Aug 07 19:31:20 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131080, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:20.397565 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.378786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:55909 (local address 127.8.216.190:35515)
0807 19:31:20.379201 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:20.379205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:20.380262 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 19:31:20.382104 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:20.382109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:20.382214 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:20.382331 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:20.383877 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.384647 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.386227 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.386773 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.386913 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:20.390848 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:20.390853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:20.390856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:20.390889 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:20.393845 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.394863 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.394867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.394869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.395007 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.395507 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.395510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.395512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.395775 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:20.395888 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:20.396873 (+ 985us) server_negotiation.cc:299] Negotiation successful
0807 19:31:20.397083 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":206,"threads_started":1}
I20260807 19:31:20.398619 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.379351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35515 (local address 127.8.216.130:55909)
0807 19:31:20.380061 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:20.380097 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:20.381834 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:20.382579 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:20.382587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:20.383121 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:20.383681 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.383698 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.384797 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.384803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:20.386055 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.386067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.387001 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.387008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:20.387252 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:20.388032 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:20.388055 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:20.390607 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:20.394003 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:20.394016 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:20.394039 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:20.394729 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:20.395118 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:20.395125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:20.395131 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:20.395391 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:20.395910 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:20.395921 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:20.396451 (+ 530us) client_negotiation.cc:769] Sending connection context
0807 19:31:20.396779 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 19:31:20.397331 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":286,"threads_started":1}
I20260807 19:31:20.400162 13841 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35515
I20260807 19:31:20.400487 13841 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:20.401160 13841 heartbeater.cc:511] Master 127.8.216.190:35515 requested a full tablet report, sending...
I20260807 19:31:20.402788 13497 ts_manager.cc:194] Registered new tserver with Master: 03ded436a38a4a17a349ce36b40edd09 (127.8.216.130:33175)
I20260807 19:31:20.403630 13497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:55909
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:20.439759 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:20.778236 13848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:20.778578 13848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:20.778759 13848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:20.786168 13848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:20.786336 13848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:20.786537 13848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:20.797075 13848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:20.798506 13848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:20.799857 13848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:20.813225 13853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:20.813194 13856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:20.813881 13854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:20.814455 13848 server_base.cc:1034] running on GCE node
I20260807 19:31:20.815069 13848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:20.816061 13848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:20.817317 13848 hybrid_clock.cc:648] HybridClock initialized: now 1786131080817275 us; error 38 us; skew 500 ppm
I20260807 19:31:20.817662 13848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:20 dist-test-slave-8w9v krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131080, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:20.820921 13848 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:20.822007 13848 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:20.822114 13848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:20.822386 13848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:20.825419 13848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "b8b124238ebd42088dd76982ebb3f3b0"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "26784031ba810fffd0806ce0032a46a0"
server_key_iv: "0d9c98990109282adccee21a96b9b857"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.826184 13848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "b8b124238ebd42088dd76982ebb3f3b0"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "26784031ba810fffd0806ce0032a46a0"
server_key_iv: "0d9c98990109282adccee21a96b9b857"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.831189 13848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:31:20.834610 13863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.835916 13848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:20.836113 13848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "b8b124238ebd42088dd76982ebb3f3b0"
format_stamp: "Formatted at 2026-08-07 19:31:20 on dist-test-slave-8w9v"
server_key: "26784031ba810fffd0806ce0032a46a0"
server_key_iv: "0d9c98990109282adccee21a96b9b857"
server_key_version: "encryptionkey@0"
I20260807 19:31:20.836386 13848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:20.864234 13848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:20.868820 13848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:20.869113 13848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:20.870082 13848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:20.871490 13848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:20.871578 13848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.871726 13848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:20.871820 13848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:20.886082 13698 heartbeater.cc:503] Master 127.8.216.190:35515 was elected leader, sending a full tablet report...
I20260807 19:31:20.906414 13848 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:39639
I20260807 19:31:20.906455 13982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:39639 every 8 connection(s)
I20260807 19:31:20.907300 13848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:20.913780 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 13848
I20260807 19:31:20.913983 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:20.914548 9058 external_mini_cluster.cc:1428] Setting key 0c526a1b90ab25d5faaa46ca29006c8a
Aug 07 19:31:20 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131080, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:20.927807 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.909827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:58255 (local address 127.8.216.190:35515)
0807 19:31:20.910361 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:20.910368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:20.911255 (+ 887us) server_negotiation.cc:407] Connection header received
0807 19:31:20.913176 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:20.913181 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:20.913287 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:20.913403 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:20.915468 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.916501 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.918493 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.918901 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.919009 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:20.922510 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:20.922515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:20.922518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:20.922550 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:20.925102 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.925816 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.925819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.925821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.925952 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.926321 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.926324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.926326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.926588 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:20.926707 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:20.927358 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 19:31:20.927534 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":235,"threads_started":1}
I20260807 19:31:20.929018 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.910352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35515 (local address 127.8.216.131:58255)
0807 19:31:20.911091 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:20.911129 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:20.912884 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:20.913701 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:20.913712 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:20.914497 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:20.915276 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.915295 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.916667 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.916672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:20.918347 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.918367 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.919101 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.919106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:20.919292 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:20.920054 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:20.920085 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:20.922280 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:20.925237 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:20.925244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:20.925258 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:20.925711 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:20.926046 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:20.926049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:20.926052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:20.926229 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:20.926699 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:20.926705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:20.927033 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 19:31:20.927275 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 19:31:20.927671 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":286,"threads_started":1}
I20260807 19:31:20.930459 13983 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35515
I20260807 19:31:20.930758 13983 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:20.931392 13983 heartbeater.cc:511] Master 127.8.216.190:35515 requested a full tablet report, sending...
I20260807 19:31:20.932924 13496 ts_manager.cc:194] Registered new tserver with Master: b8b124238ebd42088dd76982ebb3f3b0 (127.8.216.131:39639)
I20260807 19:31:20.933928 13496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:58255
I20260807 19:31:20.942907 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:31:20.959928 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.947544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.8.216.190:35515)
0807 19:31:20.947719 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:20.947724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:20.948324 (+ 600us) server_negotiation.cc:407] Connection header received
0807 19:31:20.948473 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:20.948477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:20.948547 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:20.948635 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:20.950145 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.950874 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.952157 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.952585 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.952686 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:20.954890 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:20.954894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:20.954897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:20.954929 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:20.957623 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.958226 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.958229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.958232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.958347 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:20.958654 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:20.958657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:20.958659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:20.958983 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:20.959170 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:20.959448 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 19:31:20.959624 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:31:20.964285 13496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:20.971360 13496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:21.006306 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.984329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:33175 (local address 127.0.0.1:57168)
0807 19:31:20.991616 (+ 7287us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:20.991678 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:20.991978 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:20.996247 (+ 4269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:20.996255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:20.996303 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:20.996909 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.996927 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.999785 (+ 2858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:20.999793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.003140 (+ 3347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.003153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.005198 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.005206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.005424 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.005495 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 19:31:21.005637 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:31:21.005838 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6913,"mutex_wait_us":31,"thread_start_us":11028,"threads_started":1}
I20260807 19:31:21.006469 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.984429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.8.216.130:33175)
0807 19:31:20.995416 (+ 10987us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:20.995425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:20.995461 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:31:20.995577 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:20.995585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:20.995905 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:20.997209 (+ 1304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:20.997487 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.999578 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.003319 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.004810 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.004961 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.005180 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.005962 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.006137 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10770,"thread_start_us":10637,"threads_started":1}
I20260807 19:31:21.006922 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.984417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:43453 (local address 127.0.0.1:42402)
0807 19:31:20.995601 (+ 11184us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:20.995618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:20.995803 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:20.996873 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:20.996881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:20.996919 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:20.997499 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:20.997514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.000154 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.000167 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.003466 (+ 3299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.003481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.005637 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.005644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.005815 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.005846 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:31:21.005954 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:31:21.006096 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11073,"thread_start_us":6642,"threads_started":1}
I20260807 19:31:21.007655 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.985158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.8.216.129:43453)
0807 19:31:20.994901 (+ 9743us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:20.994908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:20.995746 (+ 838us) server_negotiation.cc:407] Connection header received
0807 19:31:20.996008 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:20.996016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:20.996298 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:20.996513 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:20.997736 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:20.999871 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.003643 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.005139 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.005277 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.005469 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.006410 (+ 941us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.007341 (+ 931us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9562,"thread_start_us":9431,"threads_started":1}
I20260807 19:31:21.011785 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.996190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39639 (local address 127.0.0.1:56428)
0807 19:31:21.001512 (+ 5322us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:21.001530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:21.001648 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:21.003654 (+ 2006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:21.003658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:21.003686 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:21.004055 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.004064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.006491 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.006499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.009193 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.009205 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.010887 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.010894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.011071 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.011116 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:31:21.011216 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:31:21.011366 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5142,"thread_start_us":4946,"threads_started":1}
I20260807 19:31:21.011808 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:20.996276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.8.216.131:39639)
0807 19:31:21.002983 (+ 6707us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.002992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.003028 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:31:21.003133 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.003141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.003409 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.003634 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:21.004271 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.006276 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.009373 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.010667 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.010825 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.011066 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.011278 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.011452 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6527,"thread_start_us":6358,"threads_started":1}
I20260807 19:31:21.011198 13771 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 (DEFAULT_TABLE table=test-table [id=58ecc5756f8548518aa8fa93e60476b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:21.011198 13629 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 (DEFAULT_TABLE table=test-table [id=58ecc5756f8548518aa8fa93e60476b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:21.012797 13771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3d3387d08c43358ac296f0a4ad82d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:21.012797 13629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3d3387d08c43358ac296f0a4ad82d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:21.016281 13916 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 (DEFAULT_TABLE table=test-table [id=58ecc5756f8548518aa8fa93e60476b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:21.017786 13916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3d3387d08c43358ac296f0a4ad82d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:21.027778 14003 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Bootstrap starting.
I20260807 19:31:21.029165 14004 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Bootstrap starting.
I20260807 19:31:21.032178 14003 tablet_bootstrap.cc:654] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:21.032691 14004 tablet_bootstrap.cc:654] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:21.033222 14005 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Bootstrap starting.
I20260807 19:31:21.033989 14004 log.cc:824] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:21.033975 14003 log.cc:824] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:21.036700 14005 tablet_bootstrap.cc:654] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:21.037091 14003 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: No bootstrap required, opened a new log
I20260807 19:31:21.037339 14004 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: No bootstrap required, opened a new log
I20260807 19:31:21.037360 14003 ts_tablet_manager.cc:1397] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 19:31:21.037628 14004 ts_tablet_manager.cc:1397] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:31:21.037994 14005 log.cc:824] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:21.041281 14005 tablet_bootstrap.cc:492] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: No bootstrap required, opened a new log
I20260807 19:31:21.042066 14005 ts_tablet_manager.cc:1397] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:31:21.046015 14003 raft_consensus.cc:348] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.046411 14003 raft_consensus.cc:374] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:21.046555 14003 raft_consensus.cc:729] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabb8ab4a6fd409897126494f47b0e04, State: Initialized, Role: FOLLOWER
I20260807 19:31:21.047216 14003 consensus_queue.cc:260] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.048056 14004 raft_consensus.cc:348] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.048559 14004 raft_consensus.cc:374] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:21.048676 14004 raft_consensus.cc:729] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ded436a38a4a17a349ce36b40edd09, State: Initialized, Role: FOLLOWER
I20260807 19:31:21.049110 14003 ts_tablet_manager.cc:1428] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260807 19:31:21.049257 14004 consensus_queue.cc:260] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.050434 13841 heartbeater.cc:503] Master 127.8.216.190:35515 was elected leader, sending a full tablet report...
I20260807 19:31:21.050995 14004 ts_tablet_manager.cc:1428] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260807 19:31:21.053149 14005 raft_consensus.cc:348] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.053637 14005 raft_consensus.cc:374] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:21.053745 14005 raft_consensus.cc:729] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b124238ebd42088dd76982ebb3f3b0, State: Initialized, Role: FOLLOWER
I20260807 19:31:21.054312 14005 consensus_queue.cc:260] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.055956 13983 heartbeater.cc:503] Master 127.8.216.190:35515 was elected leader, sending a full tablet report...
I20260807 19:31:21.056519 14005 ts_tablet_manager.cc:1428] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:31:21.094115 14010 raft_consensus.cc:482] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:21.094470 14010 raft_consensus.cc:504] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.096335 14010 leader_election.cc:288] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabb8ab4a6fd409897126494f47b0e04 (127.8.216.129:43453), b8b124238ebd42088dd76982ebb3f3b0 (127.8.216.131:39639)
I20260807 19:31:21.104080 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.096812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34827 (local address 127.8.216.131:39639)
0807 19:31:21.097069 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.097077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.097768 (+ 691us) server_negotiation.cc:407] Connection header received
0807 19:31:21.097900 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.097908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.097984 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.098074 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:21.098737 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.099765 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.102216 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.103029 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.103221 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.103305 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.103671 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.103797 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 19:31:21.104101 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.096750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:56747 (local address 127.8.216.129:43453)
0807 19:31:21.096973 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.096982 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.097481 (+ 499us) server_negotiation.cc:407] Connection header received
0807 19:31:21.097735 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.097742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.097832 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.097924 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:21.098544 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.099680 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.102266 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.103027 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.103236 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.103324 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.103709 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.103835 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:31:21.104321 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.096508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:43453 (local address 127.8.216.130:56747)
0807 19:31:21.097353 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:21.097372 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:21.097577 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:21.097927 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:21.097930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:21.097946 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:21.098387 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.098400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.099833 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.099841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.102084 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.102099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.103274 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.103282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.103477 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.103547 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 19:31:21.103659 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:31:21.103791 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":116,"thread_start_us":478,"threads_started":1}
I20260807 19:31:21.104354 14013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.096806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39639 (local address 127.8.216.130:34827)
0807 19:31:21.097657 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:21.097676 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:21.097783 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:21.098129 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:21.098133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:21.098152 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:21.098596 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.098605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.099891 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.099897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.102042 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:21.102058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.103313 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.103320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:21.103490 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.103531 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:31:21.103606 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 19:31:21.103754 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":48,"thread_start_us":496,"threads_started":1}
W20260807 19:31:21.104751 13699 tablet.cc:2369] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:21.105690 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3d3387d08c43358ac296f0a4ad82d6" candidate_uuid: "03ded436a38a4a17a349ce36b40edd09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b124238ebd42088dd76982ebb3f3b0" is_pre_election: true
I20260807 19:31:21.105690 13649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3d3387d08c43358ac296f0a4ad82d6" candidate_uuid: "03ded436a38a4a17a349ce36b40edd09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb8ab4a6fd409897126494f47b0e04" is_pre_election: true
I20260807 19:31:21.106292 13937 raft_consensus.cc:2454] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03ded436a38a4a17a349ce36b40edd09 in term 0.
I20260807 19:31:21.106292 13649 raft_consensus.cc:2454] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03ded436a38a4a17a349ce36b40edd09 in term 0.
I20260807 19:31:21.107064 13722 leader_election.cc:302] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03ded436a38a4a17a349ce36b40edd09, b8b124238ebd42088dd76982ebb3f3b0, fabb8ab4a6fd409897126494f47b0e04; no voters:
I20260807 19:31:21.107678 14010 raft_consensus.cc:2781] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:21.107836 14010 raft_consensus.cc:482] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:21.107946 14010 raft_consensus.cc:3037] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:21.110208 14010 raft_consensus.cc:504] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.110828 14010 leader_election.cc:288] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [CANDIDATE]: Term 1 election: Requested vote from peers fabb8ab4a6fd409897126494f47b0e04 (127.8.216.129:43453), b8b124238ebd42088dd76982ebb3f3b0 (127.8.216.131:39639)
I20260807 19:31:21.111373 13649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3d3387d08c43358ac296f0a4ad82d6" candidate_uuid: "03ded436a38a4a17a349ce36b40edd09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabb8ab4a6fd409897126494f47b0e04"
I20260807 19:31:21.111392 13937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3d3387d08c43358ac296f0a4ad82d6" candidate_uuid: "03ded436a38a4a17a349ce36b40edd09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b124238ebd42088dd76982ebb3f3b0"
I20260807 19:31:21.111600 13937 raft_consensus.cc:3037] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:21.111600 13649 raft_consensus.cc:3037] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:21.114916 13649 raft_consensus.cc:2454] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03ded436a38a4a17a349ce36b40edd09 in term 1.
I20260807 19:31:21.114909 13937 raft_consensus.cc:2454] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03ded436a38a4a17a349ce36b40edd09 in term 1.
I20260807 19:31:21.115420 13721 leader_election.cc:302] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03ded436a38a4a17a349ce36b40edd09, b8b124238ebd42088dd76982ebb3f3b0, fabb8ab4a6fd409897126494f47b0e04; no voters:
I20260807 19:31:21.115813 14010 raft_consensus.cc:2781] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:21.116345 14010 raft_consensus.cc:686] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 1 LEADER]: Becoming Leader. State: Replica: 03ded436a38a4a17a349ce36b40edd09, State: Running, Role: LEADER
I20260807 19:31:21.116919 14010 consensus_queue.cc:237] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } }
I20260807 19:31:21.121584 13493 catalog_manager.cc:5495] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03ded436a38a4a17a349ce36b40edd09 (127.8.216.130). New cstate: current_term: 1 leader_uuid: "03ded436a38a4a17a349ce36b40edd09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03ded436a38a4a17a349ce36b40edd09" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 } health_report { overall_health: UNKNOWN } } }
W20260807 19:31:21.128198 13842 tablet.cc:2369] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:21.138325 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.132795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.8.216.130:33175)
0807 19:31:21.133007 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.133013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.133135 (+ 122us) server_negotiation.cc:407] Connection header received
0807 19:31:21.133392 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.133397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.133486 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.133645 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:21.134056 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.134762 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.136107 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.136573 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.136712 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.137195 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:21.137725 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:21.137884 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.137968 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.138047 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:31:21.146835 13937 raft_consensus.cc:1261] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 1 FOLLOWER]: Refusing update from remote peer 03ded436a38a4a17a349ce36b40edd09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:21.146880 13649 raft_consensus.cc:1261] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 1 FOLLOWER]: Refusing update from remote peer 03ded436a38a4a17a349ce36b40edd09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:21.147964 14010 consensus_queue.cc:1035] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b124238ebd42088dd76982ebb3f3b0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:21.148303 14014 consensus_queue.cc:1035] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabb8ab4a6fd409897126494f47b0e04" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 43453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:21.158038 14017 mvcc.cc:205] Tried to move back new op lower bound from 7315992908367695872 to 7315992908258922496. Current Snapshot: MvccSnapshot[applied={T|T < 7315992908367695872}]
I20260807 19:31:21.159650 14019 mvcc.cc:205] Tried to move back new op lower bound from 7315992908367695872 to 7315992908258922496. Current Snapshot: MvccSnapshot[applied={T|T < 7315992908367695872}]
I20260807 19:31:21.181529 13493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:21.181845 13493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:21.185801 13493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddce4ecf028046ebb01419091d98f10a: Sending DeleteTablet for 3 replicas of tablet 4e3d3387d08c43358ac296f0a4ad82d6
I20260807 19:31:21.187002 13771 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:21 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:57168
I20260807 19:31:21.187096 13629 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:21 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:42402
I20260807 19:31:21.187496 13916 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3d3387d08c43358ac296f0a4ad82d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:21 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:56428
I20260807 19:31:21.190161 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13562
I20260807 19:31:21.199108 14030 tablet_replica.cc:330] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: stopping tablet replica
I20260807 19:31:21.199654 14028 tablet_replica.cc:330] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: stopping tablet replica
I20260807 19:31:21.199641 14030 raft_consensus.cc:2229] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:21.200078 14028 raft_consensus.cc:2229] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:21.200217 14030 raft_consensus.cc:2258] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:21.200748 14029 tablet_replica.cc:330] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: stopping tablet replica
I20260807 19:31:21.201227 14029 raft_consensus.cc:2229] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:21.201776 14029 raft_consensus.cc:2258] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:21.202524 14028 raft_consensus.cc:2258] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:21.202713 14030 ts_tablet_manager.cc:1905] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:21.204855 14028 ts_tablet_manager.cc:1905] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:21.204873 14029 ts_tablet_manager.cc:1905] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:21.209980 14030 ts_tablet_manager.cc:1918] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:21.210242 14030 log.cc:1196] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/4e3d3387d08c43358ac296f0a4ad82d6
I20260807 19:31:21.210922 14030 ts_tablet_manager.cc:1939] T 4e3d3387d08c43358ac296f0a4ad82d6 P b8b124238ebd42088dd76982ebb3f3b0: Deleting consensus metadata
Aug 07 19:31:21 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:21.212610 14028 ts_tablet_manager.cc:1918] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:21.212891 14028 log.cc:1196] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/4e3d3387d08c43358ac296f0a4ad82d6
I20260807 19:31:21.213343 13481 catalog_manager.cc:4841] TS b8b124238ebd42088dd76982ebb3f3b0 (127.8.216.131:39639): tablet 4e3d3387d08c43358ac296f0a4ad82d6 (table test-table [id=58ecc5756f8548518aa8fa93e60476b7]) successfully deleted
I20260807 19:31:21.213593 14028 ts_tablet_manager.cc:1939] T 4e3d3387d08c43358ac296f0a4ad82d6 P 03ded436a38a4a17a349ce36b40edd09: Deleting consensus metadata
I20260807 19:31:21.215199 13481 catalog_manager.cc:4841] TS 03ded436a38a4a17a349ce36b40edd09 (127.8.216.130:33175): tablet 4e3d3387d08c43358ac296f0a4ad82d6 (table test-table [id=58ecc5756f8548518aa8fa93e60476b7]) successfully deleted
I20260807 19:31:21.216946 14029 ts_tablet_manager.cc:1918] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:21.217159 14029 log.cc:1196] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/4e3d3387d08c43358ac296f0a4ad82d6
I20260807 19:31:21.217650 14029 ts_tablet_manager.cc:1939] T 4e3d3387d08c43358ac296f0a4ad82d6 P fabb8ab4a6fd409897126494f47b0e04: Deleting consensus metadata
I20260807 19:31:21.219463 13480 catalog_manager.cc:4841] TS fabb8ab4a6fd409897126494f47b0e04 (127.8.216.129:43453): tablet 4e3d3387d08c43358ac296f0a4ad82d6 (table test-table [id=58ecc5756f8548518aa8fa93e60476b7]) successfully deleted
I20260807 19:31:21.221244 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.191996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.8.216.129:43453)
0807 19:31:21.192246 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.192255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.200630 (+ 8375us) server_negotiation.cc:407] Connection header received
0807 19:31:21.201155 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.201160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.201261 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.201354 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:21.203774 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.204906 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.206837 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.207536 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.207640 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.213268 (+ 5628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:21.213285 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:21.213305 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:21.213351 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:21.217709 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.218616 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.218627 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.218630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.218838 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.219422 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.219427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.219430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.219967 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:21.220341 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.220593 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.220842 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:31:21.221982 13693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:21.361308 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13562
I20260807 19:31:21.391444 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13704
Aug 07 19:31:21 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:21.418138 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.393185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57180 (local address 127.8.216.130:33175)
0807 19:31:21.393384 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.393390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.393411 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:31:21.393476 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.393482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.393565 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.393665 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:21.400229 (+ 6564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.401545 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.403498 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.404236 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.404358 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.410165 (+ 5807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:21.410181 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:21.410200 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:21.410253 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:21.414649 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.415620 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.415634 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.415638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.415808 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.416345 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.416350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.416354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.416802 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:21.417151 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.417486 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.417725 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:31:21.418955 13835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:21.534106 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13704
I20260807 19:31:21.552069 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13848
Aug 07 19:31:21 dist-test-slave-8w9v krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:21.566881 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:21.553037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.8.216.131:39639)
0807 19:31:21.553227 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:21.553232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:21.553251 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:31:21.553371 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:21.553375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:21.553451 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:21.553547 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:21.554810 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.555564 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.556889 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:21.557298 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:21.557394 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:21.561710 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:21.561724 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:21.561740 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:21.561773 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:21.564496 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.565103 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.565110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.565112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.565237 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:21.565552 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:21.565555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:21.565557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:21.565889 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:21.566112 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:21.566320 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 19:31:21.566502 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:31:21.567533 13978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:21.658552 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13848
I20260807 19:31:21.678714 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 13464
I20260807 19:31:21.679903 13530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:21.797751 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 13464
2026-08-07T19:31:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4426 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T19:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:21Z Disabled control of system clock
I20260807 19:31:21.875803 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44039
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:31:22.218930 14040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:22.219249 14040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:22.219424 14040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:22.226984 14040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:22.227109 14040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:22.227203 14040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:22.227324 14040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:22.227386 14040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:22.237633 14040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:22.239048 14040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:22.241010 14040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:22.253094 14046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:22.253190 14047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:22.253140 14050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:22.253909 14040 server_base.cc:1034] running on GCE node
I20260807 19:31:22.254725 14040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:22.256271 14040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:22.257558 14040 hybrid_clock.cc:648] HybridClock initialized: now 1786131082257512 us; error 39 us; skew 500 ppm
I20260807 19:31:22.257957 14040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:22.260619 14040 webserver.cc:460] Webserver started at http://127.8.216.190:38449/ using document root <none> and password file <none>
I20260807 19:31:22.261392 14040 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:22.261529 14040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:22.261869 14040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:22.265044 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "2b462d2bd65d412fa9c5d04e5d5c378d"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "730caeada0d4b83dbd87d8b7cfc52a68"
server_key_iv: "9ebaa4cf2eb45012974aaafe71d0e98f"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.265903 14040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "2b462d2bd65d412fa9c5d04e5d5c378d"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "730caeada0d4b83dbd87d8b7cfc52a68"
server_key_iv: "9ebaa4cf2eb45012974aaafe71d0e98f"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.271366 14040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:31:22.275167 14056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:22.276695 14040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:22.276903 14040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "2b462d2bd65d412fa9c5d04e5d5c378d"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "730caeada0d4b83dbd87d8b7cfc52a68"
server_key_iv: "9ebaa4cf2eb45012974aaafe71d0e98f"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.277161 14040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:22.283022 14040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:22.283841 14040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:22.284214 14040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:22.302747 14040 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:44039
I20260807 19:31:22.302796 14111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:44039 every 8 connection(s)
I20260807 19:31:22.304219 14040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:22.307533 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14040
I20260807 19:31:22.307773 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:22.308426 14112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:22.308636 9058 external_mini_cluster.cc:1428] Setting key 592684878afe921797adf29de5ef0042
I20260807 19:31:22.317070 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:22.310724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.8.216.190:44039)
0807 19:31:22.311807 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:22.311817 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:22.311865 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:31:22.313472 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:22.313493 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:22.313905 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:22.314239 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:22.315434 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:22.315440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:22.315451 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:22.315464 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:22.315529 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:22.315548 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:22.315720 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:22.315835 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 19:31:22.316186 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":437,"threads_started":1}
I20260807 19:31:22.321866 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: Bootstrap starting.
I20260807 19:31:22.324885 14112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:22.325899 14112 log.cc:824] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:22.328678 14112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: No bootstrap required, opened a new log
I20260807 19:31:22.336484 14112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } }
I20260807 19:31:22.336908 14112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:22.337030 14112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b462d2bd65d412fa9c5d04e5d5c378d, State: Initialized, Role: FOLLOWER
I20260807 19:31:22.337626 14112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } }
I20260807 19:31:22.337884 14112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:22.338060 14112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:22.338248 14112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:22.340533 14112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } }
I20260807 19:31:22.341063 14112 leader_election.cc:302] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b462d2bd65d412fa9c5d04e5d5c378d; no voters:
I20260807 19:31:22.341691 14112 leader_election.cc:288] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:22.341828 14118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:22.342609 14118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [term 1 LEADER]: Becoming Leader. State: Replica: 2b462d2bd65d412fa9c5d04e5d5c378d, State: Running, Role: LEADER
I20260807 19:31:22.343192 14118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } }
I20260807 19:31:22.344094 14112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:22.346589 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b462d2bd65d412fa9c5d04e5d5c378d. Latest consensus state: current_term: 1 leader_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } } }
I20260807 19:31:22.347227 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:22.346964 14119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b462d2bd65d412fa9c5d04e5d5c378d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44039 } } }
I20260807 19:31:22.347422 14119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:22.349246 14128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:22.354482 14128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:22.364682 14128 catalog_manager.cc:1347] Generated new cluster ID: 67d7bf6e7d62449a83ebb26469fe9647
I20260807 19:31:22.364852 14128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:22.388073 14128 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:22.388645 14128 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:22.401505 14128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: Generated new TSK 0
I20260807 19:31:22.402638 14128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:31:22.420713 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--builtin_ntp_servers=127.8.216.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:31:22.769485 14138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:22.769800 14138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:22.769964 14138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:22.777900 14138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:22.778024 14138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:22.778223 14138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:22.788085 14138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:22.789528 14138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:22.791486 14138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:22.805714 14145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:22.805857 14144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:22.805749 14147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:22.807190 14138 server_base.cc:1034] running on GCE node
I20260807 19:31:22.807816 14138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:22.808893 14138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:22.810106 14138 hybrid_clock.cc:648] HybridClock initialized: now 1786131082810062 us; error 34 us; skew 500 ppm
I20260807 19:31:22.810403 14138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:22.812844 14138 webserver.cc:460] Webserver started at http://127.8.216.129:34433/ using document root <none> and password file <none>
I20260807 19:31:22.813540 14138 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:22.813632 14138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:22.813966 14138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:22.817333 14138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "723a023b6aa94af792ed5c607dc93e66"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "c768a760477d5565b9df6f50a0d3cd8a"
server_key_iv: "5eecf51a1ed1f69fca620552753b6757"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.818229 14138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "723a023b6aa94af792ed5c607dc93e66"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "c768a760477d5565b9df6f50a0d3cd8a"
server_key_iv: "5eecf51a1ed1f69fca620552753b6757"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.823645 14138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:31:22.827378 14153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:22.828856 14138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:22.829039 14138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "723a023b6aa94af792ed5c607dc93e66"
format_stamp: "Formatted at 2026-08-07 19:31:22 on dist-test-slave-8w9v"
server_key: "c768a760477d5565b9df6f50a0d3cd8a"
server_key_iv: "5eecf51a1ed1f69fca620552753b6757"
server_key_version: "encryptionkey@0"
I20260807 19:31:22.829290 14138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:22.835071 14138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:22.835896 14138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:22.836302 14138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:22.837272 14138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:22.838735 14138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:22.838867 14138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:22.838997 14138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:22.839092 14138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:22.874456 14138 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:46577
I20260807 19:31:22.874505 14276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:46577 every 8 connection(s)
I20260807 19:31:22.875813 14138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:22.884569 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:22.878545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36671 (local address 127.8.216.190:44039)
0807 19:31:22.879045 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:22.879049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:22.880095 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 19:31:22.882153 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:22.882159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:22.882224 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:22.882354 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:22.883726 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:22.883732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:22.883736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:22.883754 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:22.883807 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:22.883819 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:22.883964 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:22.884182 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 19:31:22.884265 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":287,"threads_started":1}
I20260807 19:31:22.885529 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14138
I20260807 19:31:22.885761 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:22.885727 14280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:22.879028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44039 (local address 127.8.216.129:36671)
0807 19:31:22.879925 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:22.879970 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:22.881831 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:22.882600 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:22.882607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:22.882695 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:22.883431 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:22.883434 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:22.883471 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:22.883554 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:22.883925 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:22.883937 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:22.883947 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:31:22.884265 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:31:22.884518 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":263,"threads_started":1}
I20260807 19:31:22.886547 9058 external_mini_cluster.cc:1428] Setting key ed428d4a6d577f4f93f5457a8af9e7a0
I20260807 19:31:22.887426 14277 heartbeater.cc:347] Connected to a master server at 127.8.216.190:44039
I20260807 19:31:22.887758 14277 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:22.888522 14277 heartbeater.cc:511] Master 127.8.216.190:44039 requested a full tablet report, sending...
I20260807 19:31:22.889691 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--builtin_ntp_servers=127.8.216.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:31:22.890658 14075 ts_manager.cc:194] Registered new tserver with Master: 723a023b6aa94af792ed5c607dc93e66 (127.8.216.129:46577)
W20260807 19:31:23.235175 14281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:23.235498 14281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:23.235687 14281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:23.243319 14281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:23.243430 14281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:23.243606 14281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:23.253986 14281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:23.255357 14281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:23.257269 14281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:23.270674 14286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:23.270733 14287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:23.270759 14289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:23.271739 14281 server_base.cc:1034] running on GCE node
I20260807 19:31:23.272435 14281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:23.273360 14281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:23.274578 14281 hybrid_clock.cc:648] HybridClock initialized: now 1786131083274533 us; error 44 us; skew 500 ppm
I20260807 19:31:23.274922 14281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:23.277446 14281 webserver.cc:460] Webserver started at http://127.8.216.130:38001/ using document root <none> and password file <none>
I20260807 19:31:23.278190 14281 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:23.278316 14281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:23.278671 14281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:23.281764 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "001f4d2800526e13218a575c3d44eb1b"
server_key_iv: "da3cb629b0d3049b0afbecb29ecb3380"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.282580 14281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "001f4d2800526e13218a575c3d44eb1b"
server_key_iv: "da3cb629b0d3049b0afbecb29ecb3380"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.287992 14281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:31:23.291503 14295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.293045 14281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:23.293229 14281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "001f4d2800526e13218a575c3d44eb1b"
server_key_iv: "da3cb629b0d3049b0afbecb29ecb3380"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.293493 14281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:23.299238 14281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:23.300041 14281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:23.300343 14281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:23.301290 14281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:23.302835 14281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:23.302937 14281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.303102 14281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:23.303184 14281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.338948 14281 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:32825
I20260807 19:31:23.339006 14411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:32825 every 8 connection(s)
I20260807 19:31:23.340440 14281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:23.343076 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14281
I20260807 19:31:23.343264 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:23.343744 9058 external_mini_cluster.cc:1428] Setting key 2a3567022a7844390ba07d76176ec131
I20260807 19:31:23.347168 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--builtin_ntp_servers=127.8.216.148:42579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:31:23.349586 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.343091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:42047 (local address 127.8.216.190:44039)
0807 19:31:23.343337 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.343343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.344882 (+ 1539us) server_negotiation.cc:407] Connection header received
0807 19:31:23.346996 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.347000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.347063 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.347147 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.348796 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.348801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.348804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.348820 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.348853 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.348865 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.348960 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.349295 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.349351 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:31:23.350818 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.343593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44039 (local address 127.8.216.130:42047)
0807 19:31:23.344707 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:23.344754 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:23.346697 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:23.347450 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:23.347464 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:23.347582 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:23.348375 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:23.348382 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:23.348444 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:23.348559 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:23.348968 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:23.348981 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:23.348991 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:31:23.349352 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 19:31:23.349616 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":344,"threads_started":1}
I20260807 19:31:23.352234 14412 heartbeater.cc:347] Connected to a master server at 127.8.216.190:44039
I20260807 19:31:23.352555 14412 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:23.353149 14412 heartbeater.cc:511] Master 127.8.216.190:44039 requested a full tablet report, sending...
I20260807 19:31:23.354609 14075 ts_manager.cc:194] Registered new tserver with Master: 4cf9a76c4ca04ceeb3a11a98d41e60c2 (127.8.216.130:32825)
W20260807 19:31:23.687808 14415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:23.688140 14415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:23.688301 14415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:23.697136 14415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:23.697260 14415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:23.697381 14415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:23.707358 14415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:44039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:23.708721 14415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:23.710606 14415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:23.724584 14421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:23.724660 14423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:23.724623 14420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:23.725452 14415 server_base.cc:1034] running on GCE node
I20260807 19:31:23.726035 14415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:23.726965 14415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:23.728215 14415 hybrid_clock.cc:648] HybridClock initialized: now 1786131083728133 us; error 58 us; skew 500 ppm
I20260807 19:31:23.728562 14415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:23.730814 14415 webserver.cc:460] Webserver started at http://127.8.216.131:37829/ using document root <none> and password file <none>
I20260807 19:31:23.731732 14415 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:23.731856 14415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:23.732301 14415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:23.735374 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "2ff1c82081bf43e389dbdc1c04b5dc72"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "8ec3e367bc74bdc66ce9ed5ddd7dbde7"
server_key_iv: "a0798b2c00d49d48fc1baae9a2bf0717"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.736266 14415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "2ff1c82081bf43e389dbdc1c04b5dc72"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "8ec3e367bc74bdc66ce9ed5ddd7dbde7"
server_key_iv: "a0798b2c00d49d48fc1baae9a2bf0717"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.741670 14415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:23.745527 14429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.747390 14415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:31:23.747582 14415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "2ff1c82081bf43e389dbdc1c04b5dc72"
format_stamp: "Formatted at 2026-08-07 19:31:23 on dist-test-slave-8w9v"
server_key: "8ec3e367bc74bdc66ce9ed5ddd7dbde7"
server_key_iv: "a0798b2c00d49d48fc1baae9a2bf0717"
server_key_version: "encryptionkey@0"
I20260807 19:31:23.747869 14415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:23.754864 14415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:23.755640 14415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:23.755877 14415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:23.756850 14415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:23.758206 14415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:23.758296 14415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.758380 14415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:23.758436 14415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:23.791579 14415 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:36401
I20260807 19:31:23.791675 14558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:36401 every 8 connection(s)
I20260807 19:31:23.793061 14415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:23.801522 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.795604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48929 (local address 127.8.216.190:44039)
0807 19:31:23.795832 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.795838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.797377 (+ 1539us) server_negotiation.cc:407] Connection header received
0807 19:31:23.799109 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.799113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.799170 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.799262 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.800618 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.800626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.800632 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.800656 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.800705 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.800737 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.800883 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.801113 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.801191 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:31:23.802177 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14415
I20260807 19:31:23.802376 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:23.802556 14561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.796193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44039 (local address 127.8.216.131:48929)
0807 19:31:23.797191 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:23.797265 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:23.798867 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:23.799506 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:23.799514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:23.799600 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:23.800293 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:23.800295 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:23.800332 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:23.800437 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:23.800866 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:23.800879 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:23.800891 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:31:23.801211 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 19:31:23.801513 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":260,"threads_started":1}
I20260807 19:31:23.802943 9058 external_mini_cluster.cc:1428] Setting key a4e9c94d965e97ec46c3c777f75797cd
I20260807 19:31:23.803996 14559 heartbeater.cc:347] Connected to a master server at 127.8.216.190:44039
I20260807 19:31:23.804356 14559 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:23.805111 14559 heartbeater.cc:511] Master 127.8.216.190:44039 requested a full tablet report, sending...
I20260807 19:31:23.806507 14075 ts_manager.cc:194] Registered new tserver with Master: 2ff1c82081bf43e389dbdc1c04b5dc72 (127.8.216.131:36401)
I20260807 19:31:23.818681 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:31:23.825762 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.823338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.8.216.190:44039)
0807 19:31:23.823597 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.823606 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.823991 (+ 385us) server_negotiation.cc:407] Connection header received
0807 19:31:23.824177 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.824181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.824237 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.824320 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.825160 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.825164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.825167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.825182 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.825213 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.825225 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.825323 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.825400 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.825450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:31:23.829705 14075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:23.836979 14075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:23.865043 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.849863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:32825 (local address 127.0.0.1:41234)
0807 19:31:23.859805 (+ 9942us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:23.859832 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:23.859973 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:23.862007 (+ 2034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:23.862019 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:23.862151 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:23.863262 (+ 1111us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:23.863267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:23.863304 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:23.863436 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:23.864161 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:23.864176 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:23.864187 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:31:23.864403 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 19:31:23.864509 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9736,"thread_start_us":2244,"threads_started":1}
I20260807 19:31:23.865249 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.850628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.8.216.130:32825)
0807 19:31:23.859333 (+ 8705us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.859347 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.860146 (+ 799us) server_negotiation.cc:407] Connection header received
0807 19:31:23.860246 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.860258 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.860549 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.860727 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.863720 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.863735 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.863759 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.863799 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.863883 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.863911 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.864068 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.864817 (+ 749us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.864930 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8471,"thread_start_us":8310,"threads_started":1}
I20260807 19:31:23.866209 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.849642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:36401 (local address 127.0.0.1:56444)
0807 19:31:23.852668 (+ 3026us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:23.859518 (+ 6850us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:23.859700 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:23.862691 (+ 2991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:23.862699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:23.862736 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:23.863955 (+ 1219us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:23.863958 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:23.863974 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:23.864022 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:23.864628 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:23.864632 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:23.864636 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 19:31:23.864785 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:31:23.864845 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"thread_start_us":14493,"threads_started":1}
I20260807 19:31:23.866662 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.849966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.8.216.131:36401)
0807 19:31:23.851784 (+ 1818us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.851793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.859665 (+ 7872us) server_negotiation.cc:407] Connection header received
0807 19:31:23.859847 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.859872 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.860237 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.860428 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.864276 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.864285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.864298 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.864321 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.864379 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.864401 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.864574 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.865143 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.865244 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1646,"thread_start_us":1188,"threads_started":1}
I20260807 19:31:23.868181 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.852351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51282)
0807 19:31:23.864483 (+ 12132us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:23.864502 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:23.864620 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:23.866087 (+ 1467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:23.866096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:23.866120 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:23.866908 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:23.866913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:23.866937 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:23.866985 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:23.867472 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:23.867477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:23.867482 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:31:23.867622 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:31:23.867680 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11977,"thread_start_us":7063,"threads_started":1}
I20260807 19:31:23.868650 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:23.852925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.8.216.129:46577)
0807 19:31:23.863765 (+ 10840us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:23.863774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:23.864641 (+ 867us) server_negotiation.cc:407] Connection header received
0807 19:31:23.864737 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:23.864747 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:23.865020 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:23.865196 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:23.867135 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:23.867143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:23.867156 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:23.867177 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:23.867241 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:23.867266 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:23.867412 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:23.868191 (+ 779us) server_negotiation.cc:299] Negotiation successful
0807 19:31:23.868306 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10644,"thread_start_us":10844,"threads_started":1}
I20260807 19:31:23.869763 14346 tablet_service.cc:1467] Processing CreateTablet for tablet 28648999623d449fb0055d725a3417cf (DEFAULT_TABLE table=test-table [id=6ea4ebca64a14c0fae1959205c9f9ba1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:23.870468 14483 tablet_service.cc:1467] Processing CreateTablet for tablet 28648999623d449fb0055d725a3417cf (DEFAULT_TABLE table=test-table [id=6ea4ebca64a14c0fae1959205c9f9ba1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:23.871344 14346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28648999623d449fb0055d725a3417cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:23.871711 14483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28648999623d449fb0055d725a3417cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:23.872103 14209 tablet_service.cc:1467] Processing CreateTablet for tablet 28648999623d449fb0055d725a3417cf (DEFAULT_TABLE table=test-table [id=6ea4ebca64a14c0fae1959205c9f9ba1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:23.873449 14209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28648999623d449fb0055d725a3417cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:23.886447 14578 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Bootstrap starting.
I20260807 19:31:23.890844 14578 tablet_bootstrap.cc:654] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:23.892582 14578 log.cc:824] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:23.893720 14277 heartbeater.cc:503] Master 127.8.216.190:44039 was elected leader, sending a full tablet report...
I20260807 19:31:23.895442 14580 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Bootstrap starting.
I20260807 19:31:23.898054 14578 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: No bootstrap required, opened a new log
I20260807 19:31:23.898419 14578 ts_tablet_manager.cc:1397] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260807 19:31:23.900684 14580 tablet_bootstrap.cc:654] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:23.901901 14580 log.cc:824] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:23.903007 14579 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Bootstrap starting.
I20260807 19:31:23.906453 14580 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: No bootstrap required, opened a new log
I20260807 19:31:23.906754 14580 ts_tablet_manager.cc:1397] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:31:23.906785 14579 tablet_bootstrap.cc:654] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:23.908059 14579 log.cc:824] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:23.908380 14578 raft_consensus.cc:348] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.908947 14578 raft_consensus.cc:374] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:23.909065 14578 raft_consensus.cc:729] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cf9a76c4ca04ceeb3a11a98d41e60c2, State: Initialized, Role: FOLLOWER
I20260807 19:31:23.909760 14578 consensus_queue.cc:260] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.910908 14412 heartbeater.cc:503] Master 127.8.216.190:44039 was elected leader, sending a full tablet report...
I20260807 19:31:23.911427 14579 tablet_bootstrap.cc:492] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: No bootstrap required, opened a new log
I20260807 19:31:23.911604 14578 ts_tablet_manager.cc:1428] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 19:31:23.911713 14579 ts_tablet_manager.cc:1397] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:31:23.918666 14580 raft_consensus.cc:348] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.919209 14580 raft_consensus.cc:374] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:23.919340 14580 raft_consensus.cc:729] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723a023b6aa94af792ed5c607dc93e66, State: Initialized, Role: FOLLOWER
I20260807 19:31:23.919216 14579 raft_consensus.cc:348] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.919590 14579 raft_consensus.cc:374] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:23.919708 14579 raft_consensus.cc:729] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ff1c82081bf43e389dbdc1c04b5dc72, State: Initialized, Role: FOLLOWER
I20260807 19:31:23.920083 14580 consensus_queue.cc:260] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.920331 14579 consensus_queue.cc:260] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:23.921535 14559 heartbeater.cc:503] Master 127.8.216.190:44039 was elected leader, sending a full tablet report...
I20260807 19:31:23.922119 14579 ts_tablet_manager.cc:1428] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
I20260807 19:31:23.922130 14580 ts_tablet_manager.cc:1428] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260807 19:31:24.044999 14560 tablet.cc:2369] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:24.068863 14585 raft_consensus.cc:482] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:24.069151 14585 raft_consensus.cc:504] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:24.070997 14585 leader_election.cc:288] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ff1c82081bf43e389dbdc1c04b5dc72 (127.8.216.131:36401), 4cf9a76c4ca04ceeb3a11a98d41e60c2 (127.8.216.130:32825)
I20260807 19:31:24.074692 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.071381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:39803 (local address 127.8.216.131:36401)
0807 19:31:24.071618 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.071626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.072461 (+ 835us) server_negotiation.cc:407] Connection header received
0807 19:31:24.072678 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.072683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.072762 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.072868 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.074045 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.074051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.074055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.074076 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.074115 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.074131 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.074258 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.074351 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.074414 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 19:31:24.074692 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.071605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44707 (local address 127.8.216.130:32825)
0807 19:31:24.071818 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.071826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.072781 (+ 955us) server_negotiation.cc:407] Connection header received
0807 19:31:24.072923 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.072927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.072985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.073074 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.074055 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.074060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.074064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.074081 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.074115 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.074131 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.074260 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.074357 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.074414 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:31:24.075062 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.071547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:32825 (local address 127.8.216.129:44707)
0807 19:31:24.072678 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.072692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:24.072788 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:24.073140 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:24.073144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:24.073167 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:24.073872 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:24.073875 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:24.073888 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:24.073923 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:24.074237 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:24.074242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:24.074247 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:31:24.074406 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 19:31:24.074504 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":24,"thread_start_us":805,"threads_started":1}
I20260807 19:31:24.075102 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.071493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:36401 (local address 127.8.216.129:39803)
0807 19:31:24.072339 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.072356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:24.072529 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:24.072935 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:24.072945 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:24.072964 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:24.073786 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:31:24.073791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:24.073814 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:31:24.073873 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:24.074237 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:24.074242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:24.074247 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:31:24.074430 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 19:31:24.074491 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":467,"threads_started":1}
I20260807 19:31:24.076076 14504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28648999623d449fb0055d725a3417cf" candidate_uuid: "723a023b6aa94af792ed5c607dc93e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" is_pre_election: true
I20260807 19:31:24.076082 14367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28648999623d449fb0055d725a3417cf" candidate_uuid: "723a023b6aa94af792ed5c607dc93e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" is_pre_election: true
I20260807 19:31:24.076632 14367 raft_consensus.cc:2454] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 723a023b6aa94af792ed5c607dc93e66 in term 0.
I20260807 19:31:24.076643 14504 raft_consensus.cc:2454] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 723a023b6aa94af792ed5c607dc93e66 in term 0.
I20260807 19:31:24.077301 14158 leader_election.cc:302] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ff1c82081bf43e389dbdc1c04b5dc72, 4cf9a76c4ca04ceeb3a11a98d41e60c2, 723a023b6aa94af792ed5c607dc93e66; no voters:
I20260807 19:31:24.077704 14585 raft_consensus.cc:2781] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:24.077858 14585 raft_consensus.cc:482] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:24.077970 14585 raft_consensus.cc:3037] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:24.080330 14585 raft_consensus.cc:504] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
I20260807 19:31:24.081095 14585 leader_election.cc:288] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [CANDIDATE]: Term 1 election: Requested vote from peers 2ff1c82081bf43e389dbdc1c04b5dc72 (127.8.216.131:36401), 4cf9a76c4ca04ceeb3a11a98d41e60c2 (127.8.216.130:32825)
I20260807 19:31:24.081609 14504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28648999623d449fb0055d725a3417cf" candidate_uuid: "723a023b6aa94af792ed5c607dc93e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72"
I20260807 19:31:24.081712 14367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28648999623d449fb0055d725a3417cf" candidate_uuid: "723a023b6aa94af792ed5c607dc93e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2"
I20260807 19:31:24.081869 14504 raft_consensus.cc:3037] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:24.081920 14367 raft_consensus.cc:3037] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:24.085287 14504 raft_consensus.cc:2454] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 723a023b6aa94af792ed5c607dc93e66 in term 1.
I20260807 19:31:24.085287 14367 raft_consensus.cc:2454] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 723a023b6aa94af792ed5c607dc93e66 in term 1.
I20260807 19:31:24.085796 14155 leader_election.cc:302] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ff1c82081bf43e389dbdc1c04b5dc72, 4cf9a76c4ca04ceeb3a11a98d41e60c2, 723a023b6aa94af792ed5c607dc93e66; no voters:
I20260807 19:31:24.086210 14585 raft_consensus.cc:2781] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:24.086743 14585 raft_consensus.cc:686] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 1 LEADER]: Becoming Leader. State: Replica: 723a023b6aa94af792ed5c607dc93e66, State: Running, Role: LEADER
I20260807 19:31:24.087268 14585 consensus_queue.cc:237] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } }
W20260807 19:31:24.092496 14413 tablet.cc:2369] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:24.092077 14075 catalog_manager.cc:5495] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 723a023b6aa94af792ed5c607dc93e66 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "723a023b6aa94af792ed5c607dc93e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "723a023b6aa94af792ed5c607dc93e66" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46577 } health_report { overall_health: HEALTHY } } }
W20260807 19:31:24.128654 14278 tablet.cc:2369] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:24.158277 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.156008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.8.216.129:46577)
0807 19:31:24.156248 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.156255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.156274 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:31:24.156516 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.156520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.156602 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.156717 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.157609 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.157614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.157619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.157644 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.157676 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.157687 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.157806 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.157938 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.158002 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:31:24.166558 14504 raft_consensus.cc:1261] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 1 FOLLOWER]: Refusing update from remote peer 723a023b6aa94af792ed5c607dc93e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:24.166559 14367 raft_consensus.cc:1261] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 1 FOLLOWER]: Refusing update from remote peer 723a023b6aa94af792ed5c607dc93e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:24.167606 14589 consensus_queue.cc:1035] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ff1c82081bf43e389dbdc1c04b5dc72" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 36401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:24.167982 14585 consensus_queue.cc:1035] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cf9a76c4ca04ceeb3a11a98d41e60c2" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 32825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:24.192852 14075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:24.193181 14075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:24.197046 14075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b462d2bd65d412fa9c5d04e5d5c378d: Sending DeleteTablet for 3 replicas of tablet 28648999623d449fb0055d725a3417cf
I20260807 19:31:24.198278 14483 tablet_service.cc:1514] Processing DeleteTablet for tablet 28648999623d449fb0055d725a3417cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:24 UTC) from {username='slave'} at 127.0.0.1:56444
I20260807 19:31:24.198379 14346 tablet_service.cc:1514] Processing DeleteTablet for tablet 28648999623d449fb0055d725a3417cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:24 UTC) from {username='slave'} at 127.0.0.1:41234
I20260807 19:31:24.198726 14209 tablet_service.cc:1514] Processing DeleteTablet for tablet 28648999623d449fb0055d725a3417cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:24 UTC) from {username='slave'} at 127.0.0.1:51282
I20260807 19:31:24.200507 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14138
I20260807 19:31:24.229585 14602 tablet_replica.cc:330] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: stopping tablet replica
I20260807 19:31:24.229645 14603 tablet_replica.cc:330] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: stopping tablet replica
I20260807 19:31:24.230171 14604 tablet_replica.cc:330] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66: stopping tablet replica
I20260807 19:31:24.230157 14602 raft_consensus.cc:2229] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:24.230479 14603 raft_consensus.cc:2229] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:24.230615 14604 raft_consensus.cc:2229] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:24.230751 14602 raft_consensus.cc:2258] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:24.231163 14603 raft_consensus.cc:2258] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:24.231344 14604 raft_consensus.cc:2258] T 28648999623d449fb0055d725a3417cf P 723a023b6aa94af792ed5c607dc93e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:24.232777 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.202816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.8.216.129:46577)
0807 19:31:24.203052 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.203062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.226342 (+ 23280us) server_negotiation.cc:407] Connection header received
0807 19:31:24.227082 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.227091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.227192 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.227294 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.231752 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.231758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.231763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.231793 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.231839 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.231857 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.231995 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.232365 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.232456 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:31:24.233538 14602 ts_tablet_manager.cc:1905] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:24.233965 14603 ts_tablet_manager.cc:1905] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:24.234128 14272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:24.241375 14603 ts_tablet_manager.cc:1918] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:24.241564 14603 log.cc:1196] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/28648999623d449fb0055d725a3417cf
I20260807 19:31:24.242138 14603 ts_tablet_manager.cc:1939] T 28648999623d449fb0055d725a3417cf P 4cf9a76c4ca04ceeb3a11a98d41e60c2: Deleting consensus metadata
I20260807 19:31:24.243039 14602 ts_tablet_manager.cc:1918] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:24.243206 14602 log.cc:1196] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/28648999623d449fb0055d725a3417cf
I20260807 19:31:24.243769 14602 ts_tablet_manager.cc:1939] T 28648999623d449fb0055d725a3417cf P 2ff1c82081bf43e389dbdc1c04b5dc72: Deleting consensus metadata
I20260807 19:31:24.244269 14060 catalog_manager.cc:4841] TS 4cf9a76c4ca04ceeb3a11a98d41e60c2 (127.8.216.130:32825): tablet 28648999623d449fb0055d725a3417cf (table test-table [id=6ea4ebca64a14c0fae1959205c9f9ba1]) successfully deleted
I20260807 19:31:24.246033 14058 catalog_manager.cc:4841] TS 2ff1c82081bf43e389dbdc1c04b5dc72 (127.8.216.131:36401): tablet 28648999623d449fb0055d725a3417cf (table test-table [id=6ea4ebca64a14c0fae1959205c9f9ba1]) successfully deleted
I20260807 19:31:24.342584 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14138
W20260807 19:31:24.372715 14059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:46577 (error 108)
I20260807 19:31:24.373456 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14281
I20260807 19:31:24.374890 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.374079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51296)
0807 19:31:24.374544 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.374697 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":254,"threads_started":1}
W20260807 19:31:24.375171 14059 catalog_manager.cc:4584] TS 723a023b6aa94af792ed5c607dc93e66 (127.8.216.129:46577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28648999623d449fb0055d725a3417cf: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
I20260807 19:31:24.376899 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.374650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.8.216.130:32825)
0807 19:31:24.374820 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.374826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.374844 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:31:24.374901 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.374907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.375007 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.375094 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.376261 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.376269 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.376277 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.376299 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.376363 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.376377 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.376485 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.376613 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.376661 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:31:24.377625 14407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:24.429095 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.428492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51300)
0807 19:31:24.428740 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.428885 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:31:24.430318 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.429861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51312)
0807 19:31:24.430043 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.430142 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:31:24.469302 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14281
I20260807 19:31:24.489321 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14415
I20260807 19:31:24.491154 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.490691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51324)
0807 19:31:24.490892 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.490987 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:31:24.492381 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.491884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51336)
0807 19:31:24.492038 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.492178 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 19:31:24.492750 14573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.490152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.8.216.131:36401)
0807 19:31:24.490333 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:24.490338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:24.490429 (+ 91us) server_negotiation.cc:407] Connection header received
0807 19:31:24.490618 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:24.490622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:24.490715 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:24.490836 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:24.491914 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:24.491920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:24.491925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:24.491944 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:24.491984 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:24.492002 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:24.492182 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:24.492301 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 19:31:24.492350 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:31:24.493456 14553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:24.573242 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.572682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51338)
0807 19:31:24.572978 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.573071 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 19:31:24.574407 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:24.573966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46577 (local address 127.0.0.1:51346)
0807 19:31:24.574178 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:24.574250 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 19:31:24.582022 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14415
I20260807 19:31:24.600595 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14040
I20260807 19:31:24.601941 14107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:24.714145 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14040
2026-08-07T19:31:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2914 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 19:31:24.761677 9058 mini_oidc.cc:152] Starting JWKS server
I20260807 19:31:24.764638 9058 webserver.cc:460] Webserver started at https://127.0.0.1:39811/ using document root <none> and password file <none>
I20260807 19:31:24.764948 9058 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:31:24.765547 9058 webserver.cc:460] Webserver started at http://127.0.0.1:39999/ using document root <none> and password file <none>
2026-08-07T19:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:24Z Disabled control of system clock
I20260807 19:31:24.799273 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33935
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33935
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39811/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/testchainca.cert with env {}
W20260807 19:31:25.143199 14619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:25.143538 14619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:25.143723 14619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:25.151314 14619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:25.151463 14619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:25.151535 14619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:25.151645 14619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:25.151727 14619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:31:25.151820 14619 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:31:25.151894 14619 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39811/jwks.json
I20260807 19:31:25.162256 14619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39811/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:25.163611 14619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:25.165122 14619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:25.177902 14628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:25.177878 14626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:25.178145 14625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:25.178661 14619 server_base.cc:1034] running on GCE node
I20260807 19:31:25.179713 14619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:25.181391 14619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:25.182622 14619 hybrid_clock.cc:648] HybridClock initialized: now 1786131085182586 us; error 69 us; skew 500 ppm
I20260807 19:31:25.182994 14619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:25.185429 14619 webserver.cc:460] Webserver started at http://127.8.216.190:40101/ using document root <none> and password file <none>
I20260807 19:31:25.186156 14619 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:25.186296 14619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:25.186622 14619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:25.189781 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "91159bb3d6154ca08ef0784893004734"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "9cb5b8c104834da2cd628c45cd37b676"
server_key_iv: "6a30d747db201a014991bdc0aafe0720"
server_key_version: "encryptionkey@0"
I20260807 19:31:25.190706 14619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "91159bb3d6154ca08ef0784893004734"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "9cb5b8c104834da2cd628c45cd37b676"
server_key_iv: "6a30d747db201a014991bdc0aafe0720"
server_key_version: "encryptionkey@0"
I20260807 19:31:25.196208 14619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:31:25.199863 14635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:25.201363 14619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:25.201572 14619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "91159bb3d6154ca08ef0784893004734"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "9cb5b8c104834da2cd628c45cd37b676"
server_key_iv: "6a30d747db201a014991bdc0aafe0720"
server_key_version: "encryptionkey@0"
I20260807 19:31:25.201812 14619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:25.224887 14619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:25.225803 14619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:25.226141 14619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:25.245577 14619 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33935
I20260807 19:31:25.245597 14688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33935 every 8 connection(s)
I20260807 19:31:25.247040 14619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:25.251367 14690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:25.252142 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14619
I20260807 19:31:25.252354 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:25.253233 9058 external_mini_cluster.cc:1428] Setting key b69f92eb2ea96788e748a66fe71d9c5c
I20260807 19:31:25.266767 14690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734: Bootstrap starting.
I20260807 19:31:25.269013 14693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:25.255384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.8.216.190:33935)
0807 19:31:25.256520 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:25.256531 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:25.256579 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:31:25.258759 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:25.258786 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:25.259320 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:25.259769 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:25.260446 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.261975 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.264074 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.264881 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.265097 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:25.266451 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:25.266458 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:25.266473 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:25.267096 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:25.267180 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:25.267202 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:25.267310 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:25.267548 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 19:31:25.267914 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":357,"threads_started":1}
I20260807 19:31:25.270607 14690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:25.271955 14690 log.cc:824] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:25.273604 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:31:25.275573 14690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734: No bootstrap required, opened a new log
I20260807 19:31:25.283825 14690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } }
I20260807 19:31:25.284403 14690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:25.284539 14690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91159bb3d6154ca08ef0784893004734, State: Initialized, Role: FOLLOWER
I20260807 19:31:25.285207 14690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } }
I20260807 19:31:25.285470 14690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:25.285621 14690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:25.285825 14690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:25.288066 14690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } }
I20260807 19:31:25.288657 14690 leader_election.cc:302] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91159bb3d6154ca08ef0784893004734; no voters:
I20260807 19:31:25.289287 14690 leader_election.cc:288] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:25.289474 14695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:25.290233 14695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [term 1 LEADER]: Becoming Leader. State: Replica: 91159bb3d6154ca08ef0784893004734, State: Running, Role: LEADER
I20260807 19:31:25.290807 14695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } }
I20260807 19:31:25.291692 14690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:25.295248 14697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91159bb3d6154ca08ef0784893004734. Latest consensus state: current_term: 1 leader_uuid: "91159bb3d6154ca08ef0784893004734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } } }
I20260807 19:31:25.295493 14696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91159bb3d6154ca08ef0784893004734" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91159bb3d6154ca08ef0784893004734" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33935 } } }
I20260807 19:31:25.295784 14697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:25.295821 14696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:25.298511 14704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:25.303110 14704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:25.312033 14704 catalog_manager.cc:1347] Generated new cluster ID: 5b3d7070ca2141ddb67ace762bb87d42
I20260807 19:31:25.312198 14704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:25.318883 14704 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:25.320300 14704 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:25.332684 14704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91159bb3d6154ca08ef0784893004734: Generated new TSK 0
I20260807 19:31:25.333592 14704 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:31:25.377570 14721 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 19:31:25.377882 14721 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:31:25.379976 14693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:25.356046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.8.216.190:33935)
0807 19:31:25.356357 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:25.356364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:25.356828 (+ 464us) server_negotiation.cc:407] Connection header received
0807 19:31:25.357255 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:25.357259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:25.357328 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:25.357481 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:31:25.358140 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.358879 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.361094 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.361551 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.361645 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:25.362338 (+ 693us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:31:25.379430 (+ 17092us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:31:25.379742 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51536: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":105}
W20260807 19:31:25.380175 14693 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51536: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 19:31:25.382742 14720 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:25.355878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33935 (local address 127.0.0.1:51536)
0807 19:31:25.356704 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:25.356730 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:25.356931 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:25.357518 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:25.357524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:25.357538 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:31:25.357959 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:25.357972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.359041 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.359047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:25.360946 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:25.360964 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:25.361737 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:25.361744 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:25.361958 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:25.362046 (+ 88us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:31:25.379771 (+ 17725us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:31:25.380211 (+ 440us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.216.190:33935: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":456,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.216.190:33935: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39811/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 19:31:25.384523 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14619
I20260807 19:31:25.385754 14684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:25.502185 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14619
2026-08-07T19:31:25Z chronyd exiting
I20260807 19:31:25.557441 9058 test_util.cc:175] -----------------------------------------------
I20260807 19:31:25.557571 9058 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131039239543-9058-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (801 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 19:31:25.561376 9058 mini_oidc.cc:152] Starting JWKS server
I20260807 19:31:25.563185 9058 webserver.cc:460] Webserver started at https://127.0.0.1:35923/ using document root <none> and password file <none>
I20260807 19:31:25.563396 9058 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:31:25.564029 9058 webserver.cc:460] Webserver started at http://127.0.0.1:33233/ using document root <none> and password file <none>
2026-08-07T19:31:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:25Z Disabled control of system clock
I20260807 19:31:25.597154 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42681
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:42681
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35923/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/testchainca.cert with env {}
W20260807 19:31:25.955982 14728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:25.956446 14728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:25.956622 14728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:25.964298 14728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:25.964493 14728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:25.964589 14728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:25.964692 14728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:25.964783 14728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:31:25.964886 14728 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:31:25.964974 14728 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35923/jwks.json
I20260807 19:31:25.976590 14728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:42681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35923/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:25.978053 14728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:25.979429 14728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:25.991498 14736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:25.991602 14733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:25.991564 14734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:25.992542 14728 server_base.cc:1034] running on GCE node
I20260807 19:31:25.993393 14728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:25.994939 14728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:25.996248 14728 hybrid_clock.cc:648] HybridClock initialized: now 1786131085996195 us; error 47 us; skew 500 ppm
I20260807 19:31:25.996663 14728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:25.999064 14728 webserver.cc:460] Webserver started at http://127.8.216.190:36885/ using document root <none> and password file <none>
I20260807 19:31:25.999781 14728 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:25.999904 14728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:26.000324 14728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:26.003363 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "e0aec571fe33407f9983f065d2ff269d"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "bff2f80e8c9995baea306227c2b082fa"
server_key_iv: "baade2fa16f5f35c0b977bcd89d0c805"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.004263 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "e0aec571fe33407f9983f065d2ff269d"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "bff2f80e8c9995baea306227c2b082fa"
server_key_iv: "baade2fa16f5f35c0b977bcd89d0c805"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.009629 14728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:31:26.013398 14742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:26.014940 14728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:31:26.015159 14728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "e0aec571fe33407f9983f065d2ff269d"
format_stamp: "Formatted at 2026-08-07 19:31:25 on dist-test-slave-8w9v"
server_key: "bff2f80e8c9995baea306227c2b082fa"
server_key_iv: "baade2fa16f5f35c0b977bcd89d0c805"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.015424 14728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:26.046554 14728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:26.047628 14728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:26.048007 14728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:26.068785 14728 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:42681
I20260807 19:31:26.068794 14797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:42681 every 8 connection(s)
I20260807 19:31:26.070355 14728 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:26.074513 14798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:26.080330 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14728
I20260807 19:31:26.080567 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:26.081197 9058 external_mini_cluster.cc:1428] Setting key 95d8d224a6b3bf90c01a480de89aa8d0
I20260807 19:31:26.087167 14798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d: Bootstrap starting.
I20260807 19:31:26.091328 14798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:26.092559 14798 log.cc:824] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:26.096762 14798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d: No bootstrap required, opened a new log
I20260807 19:31:26.098043 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:26.083307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.8.216.190:42681)
0807 19:31:26.084549 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:26.084574 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:26.084628 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:31:26.087025 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:26.087053 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:26.087676 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:26.088113 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:26.088743 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.090857 (+ 2114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.092301 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.093465 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.093678 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:26.095033 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:26.095043 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:26.095061 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:26.095845 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:26.095959 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:26.096001 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:26.096146 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:26.096411 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 19:31:26.096743 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":360,"threads_started":1}
I20260807 19:31:26.102124 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:31:26.105876 14798 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } }
I20260807 19:31:26.106268 14798 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:26.106380 14798 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0aec571fe33407f9983f065d2ff269d, State: Initialized, Role: FOLLOWER
I20260807 19:31:26.107029 14798 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } }
I20260807 19:31:26.107257 14798 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:26.107421 14798 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:26.107589 14798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:26.109917 14798 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } }
I20260807 19:31:26.110456 14798 leader_election.cc:302] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0aec571fe33407f9983f065d2ff269d; no voters:
I20260807 19:31:26.111076 14798 leader_election.cc:288] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:26.111286 14803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:26.112651 14803 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [term 1 LEADER]: Becoming Leader. State: Replica: e0aec571fe33407f9983f065d2ff269d, State: Running, Role: LEADER
I20260807 19:31:26.113194 14803 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } }
I20260807 19:31:26.113723 14798 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:26.116362 14804 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0aec571fe33407f9983f065d2ff269d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } } }
I20260807 19:31:26.116257 14805 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0aec571fe33407f9983f065d2ff269d. Latest consensus state: current_term: 1 leader_uuid: "e0aec571fe33407f9983f065d2ff269d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0aec571fe33407f9983f065d2ff269d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42681 } } }
I20260807 19:31:26.116755 14804 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:26.116784 14805 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:26.119994 14810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:26.125612 14810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:26.134727 14810 catalog_manager.cc:1347] Generated new cluster ID: 327b540dfcc543a5b55dc8bc178a6fdf
I20260807 19:31:26.134871 14810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:26.155823 14810 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:26.157480 14810 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:26.164788 14810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0aec571fe33407f9983f065d2ff269d: Generated new TSK 0
I20260807 19:31:26.165630 14810 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:31:26.208524 14830 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 19:31:26.208666 14830 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:31:26.209648 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:26.181663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45848 (local address 127.8.216.190:42681)
0807 19:31:26.181834 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:26.181838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:26.182151 (+ 313us) server_negotiation.cc:407] Connection header received
0807 19:31:26.182408 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:26.182413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:26.182494 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:26.182627 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:31:26.183157 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.183867 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.185920 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.186367 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.186463 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:26.187060 (+ 597us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:31:26.209086 (+ 22026us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:31:26.209435 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 19:31:26.209827 14801 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45848: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 19:31:26.210085 14829 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:26.181391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:42681 (local address 127.0.0.1:45848)
0807 19:31:26.182036 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:26.182057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:26.182214 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:26.182674 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:26.182680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:26.182693 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:31:26.182996 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:26.183008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.184012 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.184018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:26.185770 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:26.185786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.186535 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.186542 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:26.186744 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:26.186790 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:31:26.209370 (+ 22580us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:31:26.209675 (+ 305us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.216.190:42681: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35923/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":440,"threads_started":1}
I20260807 19:31:26.211403 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14728
I20260807 19:31:26.212742 14793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:26.309641 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14728
2026-08-07T19:31:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (808 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 19:31:26.369875 9058 mini_oidc.cc:152] Starting JWKS server
I20260807 19:31:26.371868 9058 webserver.cc:460] Webserver started at https://127.0.0.1:43161/ using document root <none> and password file <none>
I20260807 19:31:26.372072 9058 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:31:26.372803 9058 webserver.cc:460] Webserver started at http://127.0.0.1:36317/ using document root <none> and password file <none>
2026-08-07T19:31:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:26Z Disabled control of system clock
I20260807 19:31:26.406744 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44775
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43161/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:31:26.746073 14837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:26.746382 14837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:26.746583 14837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:26.754017 14837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:26.754168 14837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:26.754261 14837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:26.754379 14837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:26.754462 14837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:31:26.754555 14837 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:31:26.754611 14837 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43161/jwks.json
I20260807 19:31:26.764158 14837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43161/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:26.765518 14837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:26.767333 14837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:26.779894 14843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:26.779889 14845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:26.779927 14842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:26.780869 14837 server_base.cc:1034] running on GCE node
I20260807 19:31:26.781754 14837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:26.783298 14837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:26.784538 14837 hybrid_clock.cc:648] HybridClock initialized: now 1786131086784502 us; error 62 us; skew 500 ppm
I20260807 19:31:26.784922 14837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:26.787321 14837 webserver.cc:460] Webserver started at http://127.8.216.190:45845/ using document root <none> and password file <none>
I20260807 19:31:26.788089 14837 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:26.788283 14837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:26.788647 14837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:26.791704 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "dd729dc8683a4a83a90e905bcee753b7"
format_stamp: "Formatted at 2026-08-07 19:31:26 on dist-test-slave-8w9v"
server_key: "23321b716693d1662f02dbbfcd969aef"
server_key_iv: "bb73709c677a7b0bf20663ea9c29c8e0"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.792575 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "dd729dc8683a4a83a90e905bcee753b7"
format_stamp: "Formatted at 2026-08-07 19:31:26 on dist-test-slave-8w9v"
server_key: "23321b716693d1662f02dbbfcd969aef"
server_key_iv: "bb73709c677a7b0bf20663ea9c29c8e0"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.797971 14837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:26.801621 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:26.803072 14837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:26.803277 14837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "dd729dc8683a4a83a90e905bcee753b7"
format_stamp: "Formatted at 2026-08-07 19:31:26 on dist-test-slave-8w9v"
server_key: "23321b716693d1662f02dbbfcd969aef"
server_key_iv: "bb73709c677a7b0bf20663ea9c29c8e0"
server_key_version: "encryptionkey@0"
I20260807 19:31:26.803521 14837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:26.845147 14837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:26.846347 14837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:26.846736 14837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:26.866611 14837 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:44775
I20260807 19:31:26.866642 14905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:44775 every 8 connection(s)
I20260807 19:31:26.868044 14837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:26.871932 14906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:26.878417 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14837
I20260807 19:31:26.878630 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:26.879513 9058 external_mini_cluster.cc:1428] Setting key 0918315b4cb9fb4c0528f195e7bcb0c5
I20260807 19:31:26.884037 14906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7: Bootstrap starting.
I20260807 19:31:26.887375 14906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:26.888526 14906 log.cc:824] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:26.891790 14906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7: No bootstrap required, opened a new log
I20260807 19:31:26.894330 14909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:26.881779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.8.216.190:44775)
0807 19:31:26.882822 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:26.882833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:26.882878 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:31:26.884754 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:26.884779 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:26.885239 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:26.885682 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:26.886246 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.887577 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.889102 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.890000 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.890192 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:26.891410 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:26.891422 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:26.891440 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:26.892187 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:26.892318 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:26.892352 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:26.892493 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:26.892689 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 19:31:26.893026 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":352,"threads_started":1}
I20260807 19:31:26.898411 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:31:26.901199 14906 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } }
I20260807 19:31:26.901614 14906 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:26.901727 14906 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd729dc8683a4a83a90e905bcee753b7, State: Initialized, Role: FOLLOWER
I20260807 19:31:26.902294 14906 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } }
I20260807 19:31:26.902498 14906 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:26.902593 14906 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:26.902704 14906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:26.905004 14906 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } }
I20260807 19:31:26.905587 14906 leader_election.cc:302] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd729dc8683a4a83a90e905bcee753b7; no voters:
I20260807 19:31:26.906255 14906 leader_election.cc:288] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:26.906352 14911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:26.907229 14911 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [term 1 LEADER]: Becoming Leader. State: Replica: dd729dc8683a4a83a90e905bcee753b7, State: Running, Role: LEADER
I20260807 19:31:26.907809 14911 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } }
I20260807 19:31:26.908547 14906 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:26.910858 14912 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd729dc8683a4a83a90e905bcee753b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } } }
I20260807 19:31:26.911274 14912 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:26.911701 14913 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd729dc8683a4a83a90e905bcee753b7. Latest consensus state: current_term: 1 leader_uuid: "dd729dc8683a4a83a90e905bcee753b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd729dc8683a4a83a90e905bcee753b7" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44775 } } }
I20260807 19:31:26.912067 14913 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:26.914693 14917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:26.920078 14917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:26.931850 14917 catalog_manager.cc:1347] Generated new cluster ID: b3858a0efbf74c8fb598673dd19c7269
I20260807 19:31:26.931988 14917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:26.950263 14917 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:26.951640 14917 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:26.974725 14917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd729dc8683a4a83a90e905bcee753b7: Generated new TSK 0
I20260807 19:31:26.975823 14917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:31:27.003647 14909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:26.994421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.8.216.190:44775)
0807 19:31:26.994651 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:26.994659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:26.995064 (+ 405us) server_negotiation.cc:407] Connection header received
0807 19:31:26.995267 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:26.995274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:26.995381 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:26.995579 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:31:26.996059 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.997276 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.998739 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.999539 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.999686 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:27.000233 (+ 547us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:31:27.002886 (+ 2653us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:31:27.003250 (+ 364us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:31:27.003825 14909 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44714: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 19:31:27.005311 14936 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:26.994254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44775 (local address 127.0.0.1:44714)
0807 19:31:26.994918 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:26.994934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:26.995072 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:26.995598 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:26.995601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:26.995611 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:31:26.995868 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:26.995876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.997440 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.997444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:26.998544 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:26.998556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:26.999734 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:26.999739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:26.999862 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:26.999897 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:31:27.003127 (+ 3230us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:31:27.004894 (+ 1767us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.216.190:44775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43161/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":1431,"thread_start_us":353,"threads_started":1}
I20260807 19:31:27.006510 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14837
I20260807 19:31:27.007691 14901 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 19:31:27.015525 14937 webserver.cc:533] Webserver: sslize failed:
E20260807 19:31:27.015635 14937 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:31:27.104986 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14837
2026-08-07T19:31:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (795 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T19:31:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:27Z Disabled control of system clock
I20260807 19:31:27.197655 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46451
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:33023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:46451
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/testchainca.cert with env {}
W20260807 19:31:27.540424 14942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:27.540763 14942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:27.540959 14942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:27.548452 14942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:27.548614 14942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:27.548692 14942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:27.548808 14942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:27.548893 14942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:31:27.548990 14942 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:31:27.549052 14942 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 19:31:27.559576 14942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:33023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:46451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:27.561007 14942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:27.562350 14942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:27.574707 14947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:27.574783 14950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:27.574760 14948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:27.575603 14942 server_base.cc:1034] running on GCE node
I20260807 19:31:27.576500 14942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:27.577980 14942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:27.579209 14942 hybrid_clock.cc:648] HybridClock initialized: now 1786131087579121 us; error 80 us; skew 500 ppm
I20260807 19:31:27.579589 14942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:31:27.582001 14942 webserver.cc:460] Webserver started at http://127.8.216.190:42791/ using document root <none> and password file <none>
I20260807 19:31:27.582737 14942 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:27.582885 14942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:27.583215 14942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:27.586254 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "07a3108117544898b2d351f987e87711"
format_stamp: "Formatted at 2026-08-07 19:31:27 on dist-test-slave-8w9v"
server_key: "fae41cedb4b7d49a8696997336b76616"
server_key_iv: "cb6273cc8b3e1db336050b64e4acf36e"
server_key_version: "encryptionkey@0"
I20260807 19:31:27.587157 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "07a3108117544898b2d351f987e87711"
format_stamp: "Formatted at 2026-08-07 19:31:27 on dist-test-slave-8w9v"
server_key: "fae41cedb4b7d49a8696997336b76616"
server_key_iv: "cb6273cc8b3e1db336050b64e4acf36e"
server_key_version: "encryptionkey@0"
I20260807 19:31:27.592521 14942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:31:27.596279 14956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:27.597730 14942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:27.597947 14942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "07a3108117544898b2d351f987e87711"
format_stamp: "Formatted at 2026-08-07 19:31:27 on dist-test-slave-8w9v"
server_key: "fae41cedb4b7d49a8696997336b76616"
server_key_iv: "cb6273cc8b3e1db336050b64e4acf36e"
server_key_version: "encryptionkey@0"
I20260807 19:31:27.598188 14942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:27.620148 14942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:27.621053 14942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:27.621347 14942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:27.642148 14942 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:46451
I20260807 19:31:27.642179 15010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:46451 every 8 connection(s)
I20260807 19:31:27.643571 14942 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:27.647490 15011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:27.649639 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 14942
I20260807 19:31:27.649870 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:27.650506 9058 external_mini_cluster.cc:1428] Setting key d0ce36c79e9dfeb0acbcb3591c9d4c3c
I20260807 19:31:27.659353 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711: Bootstrap starting.
I20260807 19:31:27.662513 15011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:27.663686 15011 log.cc:824] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:27.663966 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:27.652536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.8.216.190:46451)
0807 19:31:27.653609 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:27.653620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:27.653670 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:31:27.655350 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:27.655372 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:27.655799 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:27.656227 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:27.656833 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.658093 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.659560 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.660282 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.660475 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:27.661686 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:27.661694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:27.661706 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:31:27.662230 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:27.662312 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:31:27.662332 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:27.662467 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:27.662707 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 19:31:27.663027 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":406,"threads_started":1}
I20260807 19:31:27.666822 15011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711: No bootstrap required, opened a new log
I20260807 19:31:27.667806 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:31:27.674594 15011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } }
I20260807 19:31:27.674968 15011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:27.675086 15011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a3108117544898b2d351f987e87711, State: Initialized, Role: FOLLOWER
I20260807 19:31:27.675689 15011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } }
I20260807 19:31:27.675932 15011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:27.676096 15011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:27.676306 15011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:27.678736 15011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } }
I20260807 19:31:27.679251 15011 leader_election.cc:302] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07a3108117544898b2d351f987e87711; no voters:
I20260807 19:31:27.679854 15011 leader_election.cc:288] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:27.680008 15016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:27.680845 15016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [term 1 LEADER]: Becoming Leader. State: Replica: 07a3108117544898b2d351f987e87711, State: Running, Role: LEADER
I20260807 19:31:27.681430 15016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } }
I20260807 19:31:27.682137 15011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:27.684522 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07a3108117544898b2d351f987e87711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } } }
I20260807 19:31:27.685140 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:27.685042 15018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07a3108117544898b2d351f987e87711. Latest consensus state: current_term: 1 leader_uuid: "07a3108117544898b2d351f987e87711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07a3108117544898b2d351f987e87711" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46451 } } }
I20260807 19:31:27.685362 15018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:27.686031 15023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:27.691511 15023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:27.701581 15023 catalog_manager.cc:1347] Generated new cluster ID: 76058442d425480e987f95cc589aba77
I20260807 19:31:27.701725 15023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:27.734917 15023 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:27.736353 15023 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:27.745170 15023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07a3108117544898b2d351f987e87711: Generated new TSK 0
I20260807 19:31:27.746407 15023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:31:27.806308 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:27.761962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.8.216.190:46451)
0807 19:31:27.762141 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:27.762146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:27.762674 (+ 528us) server_negotiation.cc:407] Connection header received
0807 19:31:27.762966 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:27.762970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:27.763041 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:27.763225 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:31:27.763737 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.764575 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.765991 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.766426 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.766567 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:27.767074 (+ 507us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:31:27.805519 (+ 38445us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:31:27.805969 (+ 450us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35152: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 19:31:27.806501 15014 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35152: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 19:31:27.806531 15041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:27.761952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:46451 (local address 127.0.0.1:35152)
0807 19:31:27.762561 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:27.762578 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:27.762752 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:27.763231 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:27.763236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:27.763248 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:31:27.763571 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:27.763580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.764733 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.764738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:27.765809 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:27.765819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:27.766608 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:27.766614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:27.766754 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:27.766787 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:31:27.805852 (+ 39065us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:31:27.806135 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.216.190:46451: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":312,"threads_started":1}
I20260807 19:31:27.808279 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 14942
I20260807 19:31:27.809516 15006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:27.909039 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 14942
2026-08-07T19:31:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (784 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (434 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (433 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:28 dist-test-slave-8w9v krb5kdc[15056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:28 dist-test-slave-8w9v krb5kdc[15056](info): set up 2 sockets
Aug 07 19:31:28 dist-test-slave-8w9v krb5kdc[15056](info): commencing operation
krb5kdc: starting...
W20260807 19:31:29.990697 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:30 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:30.236346 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42167
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:30.580932 15069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:30.581321 15069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:30.581504 15069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:30.588987 15069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:30.589184 15069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:30.589260 15069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:30.589316 15069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:30.589363 15069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:30.599575 15069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:42167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:30.600926 15069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:30.602260 15069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:30.614668 15074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:30.614717 15075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:30.614665 15077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:30.615600 15069 server_base.cc:1034] running on GCE node
I20260807 19:31:30.616514 15069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:30.617970 15069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:30.619204 15069 hybrid_clock.cc:648] HybridClock initialized: now 1786131090619154 us; error 41 us; skew 500 ppm
I20260807 19:31:30.619554 15069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:30 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131090, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:30.625901 15069 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:30.627375 15069 webserver.cc:460] Webserver started at http://127.8.216.190:34975/ using document root <none> and password file <none>
I20260807 19:31:30.628067 15069 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:30.628237 15069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:30.628522 15069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:30.631495 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "d81b1011f6774884b69f242f16c03a1f"
format_stamp: "Formatted at 2026-08-07 19:31:30 on dist-test-slave-8w9v"
server_key: "67034a55abcb0e7e9c3dbe858c41be66"
server_key_iv: "defdef1753827492798afa8c72814844"
server_key_version: "encryptionkey@0"
I20260807 19:31:30.632339 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "d81b1011f6774884b69f242f16c03a1f"
format_stamp: "Formatted at 2026-08-07 19:31:30 on dist-test-slave-8w9v"
server_key: "67034a55abcb0e7e9c3dbe858c41be66"
server_key_iv: "defdef1753827492798afa8c72814844"
server_key_version: "encryptionkey@0"
I20260807 19:31:30.637513 15069 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:31:30.641412 15085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:30.643316 15069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:31:30.643555 15069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "d81b1011f6774884b69f242f16c03a1f"
format_stamp: "Formatted at 2026-08-07 19:31:30 on dist-test-slave-8w9v"
server_key: "67034a55abcb0e7e9c3dbe858c41be66"
server_key_iv: "defdef1753827492798afa8c72814844"
server_key_version: "encryptionkey@0"
I20260807 19:31:30.643858 15069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:30.660728 15069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:30.664878 15069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:30.665154 15069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:30.684372 15069 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:42167
I20260807 19:31:30.684412 15141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:42167 every 8 connection(s)
I20260807 19:31:30.685853 15069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:30.689723 15142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:30.691242 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15069
I20260807 19:31:30.691488 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:30.692111 9058 external_mini_cluster.cc:1428] Setting key 4d29607f81e12454b61794afa66b944c
I20260807 19:31:30.703179 15142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f: Bootstrap starting.
I20260807 19:31:30.706604 15142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f: Neither blocks nor log segments found. Creating new log.
Aug 07 19:31:30 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:30.708084 15142 log.cc:824] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:30.711697 15142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f: No bootstrap required, opened a new log
I20260807 19:31:30.717144 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:30.694219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.8.216.190:42167)
0807 19:31:30.695118 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:30.695135 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:30.695189 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:31:30.697682 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:30.697711 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:30.698226 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:30.698678 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:30.699963 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:30.702105 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:30.703334 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:30.704382 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:30.704559 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:30.708531 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:30.708544 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:30.708563 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:30.708613 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:30.712532 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:30.713217 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:30.713226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:30.713234 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:30.713469 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:30.713828 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:30.713838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:30.713842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:30.714605 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:30.714916 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:30.715250 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 19:31:30.715693 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":338,"threads_started":1}
I20260807 19:31:30.721848 15142 raft_consensus.cc:348] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } }
I20260807 19:31:30.722191 15142 raft_consensus.cc:374] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:30.722267 15142 raft_consensus.cc:729] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81b1011f6774884b69f242f16c03a1f, State: Initialized, Role: FOLLOWER
I20260807 19:31:30.722838 15142 consensus_queue.cc:260] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } }
I20260807 19:31:30.723145 15142 raft_consensus.cc:388] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:30.723299 15142 raft_consensus.cc:482] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:30.723507 15142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:30.725847 15142 raft_consensus.cc:504] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } }
I20260807 19:31:30.726444 15142 leader_election.cc:302] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d81b1011f6774884b69f242f16c03a1f; no voters:
I20260807 19:31:30.727090 15142 leader_election.cc:288] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:30.727242 15147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:30.727996 15147 raft_consensus.cc:686] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [term 1 LEADER]: Becoming Leader. State: Replica: d81b1011f6774884b69f242f16c03a1f, State: Running, Role: LEADER
I20260807 19:31:30.728614 15147 consensus_queue.cc:237] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } }
I20260807 19:31:30.729247 15142 sys_catalog.cc:564] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:30.731452 15149 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d81b1011f6774884b69f242f16c03a1f. Latest consensus state: current_term: 1 leader_uuid: "d81b1011f6774884b69f242f16c03a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } } }
I20260807 19:31:30.731423 15148 sys_catalog.cc:455] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d81b1011f6774884b69f242f16c03a1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d81b1011f6774884b69f242f16c03a1f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42167 } } }
I20260807 19:31:30.731958 15148 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:30.732685 15149 sys_catalog.cc:458] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:30.737614 15153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:30.743090 15153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:30.754323 15153 catalog_manager.cc:1347] Generated new cluster ID: bd65785c28e940ceb2df49adfbccc034
I20260807 19:31:30.754467 15153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:30.776715 15153 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:30.778102 15153 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:30.788663 15153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d81b1011f6774884b69f242f16c03a1f: Generated new TSK 0
I20260807 19:31:30.789595 15153 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:30.852293 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:31.198910 15169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:31.199259 15169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:31.199430 15169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:31.206935 15169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:31.207099 15169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:31.207288 15169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:31.217388 15169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:31.218786 15169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:31.220191 15169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:31.233984 15174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:31.233944 15175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:31.233901 15177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:31.235044 15169 server_base.cc:1034] running on GCE node
I20260807 19:31:31.235668 15169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:31.236764 15169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:31.237975 15169 hybrid_clock.cc:648] HybridClock initialized: now 1786131091237905 us; error 51 us; skew 500 ppm
I20260807 19:31:31.238343 15169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:31 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:31.241775 15169 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:31.243232 15169 webserver.cc:460] Webserver started at http://127.8.216.129:39307/ using document root <none> and password file <none>
I20260807 19:31:31.243906 15169 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:31.244017 15169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:31.244352 15169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:31.247359 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "531c693d5e454c75b6e2356ba6d2afd9"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "7e2809ef1446767b9fddb8b44bb1c134"
server_key_iv: "f50aa16b0bcddfbfd294819741d65e5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.248193 15169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "531c693d5e454c75b6e2356ba6d2afd9"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "7e2809ef1446767b9fddb8b44bb1c134"
server_key_iv: "f50aa16b0bcddfbfd294819741d65e5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.253829 15169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:31:31.257567 15184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.259047 15169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:31.259197 15169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "531c693d5e454c75b6e2356ba6d2afd9"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "7e2809ef1446767b9fddb8b44bb1c134"
server_key_iv: "f50aa16b0bcddfbfd294819741d65e5e"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.259383 15169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:31.279675 15169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:31.284601 15169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:31.284865 15169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:31.285835 15169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:31.287250 15169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:31.287328 15169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.287449 15169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:31.287530 15169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.321920 15169 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:32865
I20260807 19:31:31.321966 15302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:32865 every 8 connection(s)
I20260807 19:31:31.323295 15169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:31.328662 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15169
I20260807 19:31:31.328929 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:31.329701 9058 external_mini_cluster.cc:1428] Setting key 540223c53e6c5c51b5f7929e619beb1e
Aug 07 19:31:31 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:31.345428 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:31.325997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:53757 (local address 127.8.216.190:42167)
0807 19:31:31.326462 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:31.326467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:31.327404 (+ 937us) server_negotiation.cc:407] Connection header received
0807 19:31:31.329421 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:31.329428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:31.329527 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:31.329658 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:31.331294 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.332053 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.333609 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.334029 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.334171 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:31.338364 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:31.338370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:31.338373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:31.338406 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:31.341994 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:31.342748 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:31.342756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:31.342762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:31.342906 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:31.343326 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:31.343334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:31.343339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:31.343758 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:31.343918 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:31.344665 (+ 747us) server_negotiation.cc:299] Negotiation successful
0807 19:31:31.344926 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260807 19:31:31.346087 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:31.326536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:42167 (local address 127.8.216.129:53757)
0807 19:31:31.327232 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:31.327277 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:31.329136 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:31.329900 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:31.329908 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:31.330517 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:31.331117 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:31.331134 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.332245 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.332255 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:31.333456 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:31.333469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.334268 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.334276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:31.334522 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:31.335224 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:31.335250 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:31.338097 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:31.342140 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:31.342147 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:31.342160 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:31.342630 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:31.343020 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:31.343023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:31.343025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:31.343212 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:31.343882 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:31.343888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:31.344273 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 19:31:31.344520 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 19:31:31.344923 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":270,"threads_started":1}
I20260807 19:31:31.347815 15303 heartbeater.cc:347] Connected to a master server at 127.8.216.190:42167
I20260807 19:31:31.348166 15303 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:31.349066 15303 heartbeater.cc:511] Master 127.8.216.190:42167 requested a full tablet report, sending...
I20260807 19:31:31.351331 15104 ts_manager.cc:194] Registered new tserver with Master: 531c693d5e454c75b6e2356ba6d2afd9 (127.8.216.129:32865)
I20260807 19:31:31.352870 15104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:53757
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:31.387626 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:31.722607 15309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:31.722898 15309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:31.723061 15309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:31.730540 15309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:31.730719 15309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:31.730921 15309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:31.741370 15309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:31.742694 15309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:31.744211 15309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:31.761071 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:31.761164 15315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:31.761116 15317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:31.762640 15309 server_base.cc:1034] running on GCE node
I20260807 19:31:31.763339 15309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:31.764561 15309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:31.765806 15309 hybrid_clock.cc:648] HybridClock initialized: now 1786131091765765 us; error 39 us; skew 500 ppm
I20260807 19:31:31.766179 15309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:31 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:31.769498 15309 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:31.771152 15309 webserver.cc:460] Webserver started at http://127.8.216.130:41383/ using document root <none> and password file <none>
I20260807 19:31:31.771884 15309 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:31.772040 15309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:31.772454 15309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:31.775519 15309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "608da0b696e24a0cbb69fe906eff7077"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "e07df398523992e48feb942cbdb4eb21"
server_key_iv: "67f2526a2bb6f8e3e4b88f83ff477394"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.776423 15309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "608da0b696e24a0cbb69fe906eff7077"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "e07df398523992e48feb942cbdb4eb21"
server_key_iv: "67f2526a2bb6f8e3e4b88f83ff477394"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.781687 15309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:31:31.785164 15324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.786492 15309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:31:31.786667 15309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "608da0b696e24a0cbb69fe906eff7077"
format_stamp: "Formatted at 2026-08-07 19:31:31 on dist-test-slave-8w9v"
server_key: "e07df398523992e48feb942cbdb4eb21"
server_key_iv: "67f2526a2bb6f8e3e4b88f83ff477394"
server_key_version: "encryptionkey@0"
I20260807 19:31:31.786880 15309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:31.807829 15309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:31.811996 15309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:31.812333 15309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:31.813385 15309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:31.814757 15309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:31.814867 15309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.815011 15309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:31.815081 15309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:31.849203 15309 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38457
I20260807 19:31:31.849243 15443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38457 every 8 connection(s)
I20260807 19:31:31.850801 15309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:31.854408 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15309
I20260807 19:31:31.854632 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:31.855304 9058 external_mini_cluster.cc:1428] Setting key ca57d9b27813b8cea5c1be06979ec10b
Aug 07 19:31:31 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:31.878163 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:31.855954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:39879 (local address 127.8.216.190:42167)
0807 19:31:31.859464 (+ 3510us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:31.859472 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:31.859498 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:31:31.860543 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:31.860553 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:31.860656 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:31.860824 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:31.863003 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.863745 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.865524 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.866006 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.866180 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:31.870474 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:31.870485 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:31.870493 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:31.870544 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:31.874499 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:31.875302 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:31.875312 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:31.875318 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:31.875505 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:31.875937 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:31.875945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:31.875952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:31.876444 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:31.876620 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:31.877344 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 19:31:31.877618 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3356,"thread_start_us":3401,"threads_started":1}
I20260807 19:31:31.878691 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:31.856774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:42167 (local address 127.8.216.130:39879)
0807 19:31:31.857746 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:31.857789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:31.860223 (+ 2434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:31.861276 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:31.861311 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:31.862105 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:31.862784 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:31.862814 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.863966 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.863974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:31.865359 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:31.865373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:31.866278 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:31.866286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:31.866568 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:31.867360 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:31.867392 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:31.870186 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:31.874665 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:31.874672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:31.874685 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:31.875131 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:31.875628 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:31.875631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:31.875633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:31.875812 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:31.876572 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:31.876578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:31.876917 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 19:31:31.877144 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 19:31:31.877535 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":425,"threads_started":1}
I20260807 19:31:31.880230 15444 heartbeater.cc:347] Connected to a master server at 127.8.216.190:42167
I20260807 19:31:31.880554 15444 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:31.881286 15444 heartbeater.cc:511] Master 127.8.216.190:42167 requested a full tablet report, sending...
I20260807 19:31:31.882697 15104 ts_manager.cc:194] Registered new tserver with Master: 608da0b696e24a0cbb69fe906eff7077 (127.8.216.130:38457)
I20260807 19:31:31.883510 15104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:39879
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:31.916033 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:32.252722 15451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:32.253011 15451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:32.253188 15451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:32.260560 15451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:32.260679 15451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:32.260876 15451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:32.271301 15451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:42167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:32.272662 15451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:32.274019 15451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:32.287559 15457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:32.287521 15458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:32.287587 15460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:32.288736 15451 server_base.cc:1034] running on GCE node
I20260807 19:31:32.289341 15451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:32.290343 15451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:32.291535 15451 hybrid_clock.cc:648] HybridClock initialized: now 1786131092291465 us; error 57 us; skew 500 ppm
I20260807 19:31:32.291870 15451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.295123 15451 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:32.296630 15451 webserver.cc:460] Webserver started at http://127.8.216.131:43733/ using document root <none> and password file <none>
I20260807 19:31:32.297295 15451 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:32.297395 15451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:32.297649 15451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:32.300591 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "fa0672cb14334a79b7ef8e11bbec5cf4"
format_stamp: "Formatted at 2026-08-07 19:31:32 on dist-test-slave-8w9v"
server_key: "d0631c97766603c4a0286138af3d1a60"
server_key_iv: "2cc4d7c692680bb3dd85e87d8001819a"
server_key_version: "encryptionkey@0"
I20260807 19:31:32.301296 15451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "fa0672cb14334a79b7ef8e11bbec5cf4"
format_stamp: "Formatted at 2026-08-07 19:31:32 on dist-test-slave-8w9v"
server_key: "d0631c97766603c4a0286138af3d1a60"
server_key_iv: "2cc4d7c692680bb3dd85e87d8001819a"
server_key_version: "encryptionkey@0"
I20260807 19:31:32.306371 15451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:31:32.309731 15467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:32.311044 15451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 19:31:32.311226 15451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "fa0672cb14334a79b7ef8e11bbec5cf4"
format_stamp: "Formatted at 2026-08-07 19:31:32 on dist-test-slave-8w9v"
server_key: "d0631c97766603c4a0286138af3d1a60"
server_key_iv: "2cc4d7c692680bb3dd85e87d8001819a"
server_key_version: "encryptionkey@0"
I20260807 19:31:32.311432 15451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:32.332806 15451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:32.337293 15451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:32.337699 15451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:32.338752 15451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:32.340178 15451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:32.340291 15451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:32.340447 15451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:32.340523 15451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:32.356184 15303 heartbeater.cc:503] Master 127.8.216.190:42167 was elected leader, sending a full tablet report...
I20260807 19:31:32.374810 15451 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:46689
I20260807 19:31:32.374907 15589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:46689 every 8 connection(s)
I20260807 19:31:32.376263 15451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:32.379799 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15451
I20260807 19:31:32.380048 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:32.380646 9058 external_mini_cluster.cc:1428] Setting key fa4936bd5c4c29ee8a024b128517304a
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:32.396771 15592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.378915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:45637 (local address 127.8.216.190:42167)
0807 19:31:32.379443 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.379452 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.380554 (+ 1102us) server_negotiation.cc:407] Connection header received
0807 19:31:32.382657 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.382661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.382780 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.382908 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.384591 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.385391 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.386867 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.387269 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.387359 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.390919 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.390925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.390928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.390971 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.393620 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.394371 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.394376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.394378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.394499 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.394940 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.394949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.394951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.395217 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.395399 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.396116 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.396388 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":522,"threads_started":1}
I20260807 19:31:32.397749 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.379416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:42167 (local address 127.8.216.131:45637)
0807 19:31:32.380366 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:32.380414 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:32.382351 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:32.383119 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:32.383127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:32.383758 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:32.384427 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:32.384453 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.385512 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.385515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:32.386742 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:32.386753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.387454 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.387459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:32.387649 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.388414 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:32.388441 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:32.390639 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:32.393750 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:32.393757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:32.393769 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:32.394208 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:32.394617 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:32.394623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:32.394626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:32.394808 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:32.395350 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:32.395358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:32.395711 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 19:31:32.395998 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 19:31:32.396579 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":392,"threads_started":1}
I20260807 19:31:32.399099 15590 heartbeater.cc:347] Connected to a master server at 127.8.216.190:42167
I20260807 19:31:32.399382 15590 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:32.400077 15590 heartbeater.cc:511] Master 127.8.216.190:42167 requested a full tablet report, sending...
I20260807 19:31:32.401489 15104 ts_manager.cc:194] Registered new tserver with Master: fa0672cb14334a79b7ef8e11bbec5cf4 (127.8.216.131:46689)
I20260807 19:31:32.402281 15104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:45637
I20260807 19:31:32.408538 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/test-admin.keytab.
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.439756 9058 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 19:31:32.444955 15599 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:32.442416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.8.216.129:32865)
0807 19:31:32.443036 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.443044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.443072 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:32.443135 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.443141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.443351 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.443479 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.444494 (+ 1015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56812: BlockingRecv error: recv got EOF from 127.0.0.1:56812 (error 108)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":284,"threads_started":1}
W20260807 19:31:32.445452 9058 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.447001 9058 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.448570 9058 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:32.464869 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.450400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.8.216.129:32865)
0807 19:31:32.450602 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.450607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.450693 (+ 86us) server_negotiation.cc:407] Connection header received
0807 19:31:32.450835 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.450839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.450911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.450995 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.451939 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.453008 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.454222 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.454781 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.454895 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.458519 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.458532 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.458546 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.458579 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.461881 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.462574 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.462581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.462583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.462701 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.463053 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.463056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.463058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.463480 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.463697 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.464250 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.464580 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 19:31:32.470674 15599 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:32.469086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.8.216.129:32865)
0807 19:31:32.469302 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.469306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.469398 (+ 92us) server_negotiation.cc:407] Connection header received
0807 19:31:32.469553 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.469556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.469611 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.469687 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.470417 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56828: BlockingRecv error: recv got EOF from 127.0.0.1:56828 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:31:32.470855 9058 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.472564 9058 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.473906 9058 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:32.487895 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.475235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.8.216.129:32865)
0807 19:31:32.475400 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.475404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.475836 (+ 432us) server_negotiation.cc:407] Connection header received
0807 19:31:32.475976 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.475979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.476042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.476180 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.477071 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.477771 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.478939 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.479365 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.479465 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.482995 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.483000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.483003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.483048 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.485756 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.486319 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.486323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.486325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.486448 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.486780 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.486783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.486785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.487054 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.487260 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.487491 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.487656 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 19:31:32.492746 15599 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:32.490820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.8.216.129:32865)
0807 19:31:32.491068 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.491074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.491288 (+ 214us) server_negotiation.cc:407] Connection header received
0807 19:31:32.491487 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.491494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.491572 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.491677 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.492472 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56848: BlockingRecv error: recv got EOF from 127.0.0.1:56848 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:31:32.493012 9058 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131039239543-9058-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.494588 9058 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:32.495838 9058 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:32.509939 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.497225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56860 (local address 127.8.216.129:32865)
0807 19:31:32.497459 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.497465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.497666 (+ 201us) server_negotiation.cc:407] Connection header received
0807 19:31:32.497780 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.497783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.497839 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.497920 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.498817 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.499522 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.500750 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.501182 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.501275 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.504803 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.504808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.504811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.504843 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.507544 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.508112 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.508116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.508157 (+ 41us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.508321 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.508679 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.508683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.508685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.508961 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.509189 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.509399 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.509599 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:31:32.511525 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15169
I20260807 19:31:32.523170 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.512290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.8.216.129:32865)
0807 19:31:32.512443 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.512447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.512865 (+ 418us) server_negotiation.cc:407] Connection header received
0807 19:31:32.512986 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.512990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.513044 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.513120 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.514006 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.514701 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.516046 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.516531 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.516632 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.518549 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.518554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.518557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.518585 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.521129 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.521693 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.521696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.521698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.521823 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.522121 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.522124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.522126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.522355 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.522559 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.522769 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.522923 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:31:32.523736 15298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:32.608510 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15169
I20260807 19:31:32.628269 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15309
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:32.643640 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.629177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37704 (local address 127.8.216.130:38457)
0807 19:31:32.629772 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.629779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.629804 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:31:32.629914 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.629924 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.630147 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.630282 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.631215 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.632290 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.633410 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.633950 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.634044 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.637924 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.637935 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.637950 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.637981 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.640918 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.641592 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.641603 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.641611 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.641724 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.642083 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.642088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.642092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.642435 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.642742 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.643070 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.643315 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":308,"threads_started":1}
I20260807 19:31:32.644930 15439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:32.730177 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15309
I20260807 19:31:32.748569 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15451
Aug 07 19:31:32 dist-test-slave-8w9v krb5kdc[15056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131092, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:32.765623 15614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:32.749515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.8.216.131:46689)
0807 19:31:32.750199 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:32.750206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:32.750237 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:31:32.750302 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:32.750309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:32.750516 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:32.750671 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:32.751707 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.752827 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.754145 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:32.754684 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:32.754783 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:32.759523 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:32.759535 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:32.759549 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:32.759582 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:32.762556 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.763305 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.763312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.763314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.763446 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:32.763838 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:32.763852 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:32.763859 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:32.764379 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:32.764701 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:32.765047 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 19:31:32.765317 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":363,"threads_started":1}
I20260807 19:31:32.766947 15585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:32.850303 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15451
I20260807 19:31:32.868116 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15069
I20260807 19:31:32.869424 15135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:32.977926 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15069
2026-08-07T19:31:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4208 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:33 dist-test-slave-8w9v krb5kdc[15619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:33 dist-test-slave-8w9v krb5kdc[15619](info): set up 2 sockets
Aug 07 19:31:33 dist-test-slave-8w9v krb5kdc[15619](info): commencing operation
krb5kdc: starting...
W20260807 19:31:34.130616 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:34 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab.
Entry for principal oryx/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab.
I20260807 19:31:34.383289 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:40249
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab
--principal=oryx/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:34.733043 15633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:34.733366 15633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:34.733553 15633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:34.741367 15633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:34.741499 15633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:34.741606 15633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:34.741698 15633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:34.741804 15633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:34.752447 15633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:40249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab
--principal=oryx/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:34.753952 15633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:34.755347 15633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:34.768318 15638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:34.768370 15639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:34.768496 15641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:34.769346 15633 server_base.cc:1034] running on GCE node
I20260807 19:31:34.770318 15633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:34.771967 15633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:34.773232 15633 hybrid_clock.cc:648] HybridClock initialized: now 1786131094773193 us; error 99 us; skew 500 ppm
I20260807 19:31:34.773615 15633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:34 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:34.777096 15633 init.cc:377] Logged in from keytab as oryx/127.8.216.190@KRBTEST.COM (short username oryx)
I20260807 19:31:34.778891 15633 webserver.cc:460] Webserver started at http://127.8.216.190:40591/ using document root <none> and password file <none>
I20260807 19:31:34.779706 15633 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:34.779866 15633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:34.780304 15633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:34.783591 15633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "1eca52ce8d2d4ce4a2a2f52245054831"
format_stamp: "Formatted at 2026-08-07 19:31:34 on dist-test-slave-8w9v"
server_key: "d34c5ae83177979a48d7cce71f22853f"
server_key_iv: "42b4af55b3067c96bf87da53d463caf0"
server_key_version: "encryptionkey@0"
I20260807 19:31:34.784500 15633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "1eca52ce8d2d4ce4a2a2f52245054831"
format_stamp: "Formatted at 2026-08-07 19:31:34 on dist-test-slave-8w9v"
server_key: "d34c5ae83177979a48d7cce71f22853f"
server_key_iv: "42b4af55b3067c96bf87da53d463caf0"
server_key_version: "encryptionkey@0"
I20260807 19:31:34.790190 15633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:31:34.793924 15649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:34.795320 15633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:31:34.795550 15633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "1eca52ce8d2d4ce4a2a2f52245054831"
format_stamp: "Formatted at 2026-08-07 19:31:34 on dist-test-slave-8w9v"
server_key: "d34c5ae83177979a48d7cce71f22853f"
server_key_iv: "42b4af55b3067c96bf87da53d463caf0"
server_key_version: "encryptionkey@0"
I20260807 19:31:34.795786 15633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:34.826531 15633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:34.830848 15633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:34.831199 15633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:34.850612 15633 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:40249
I20260807 19:31:34.850644 15702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:40249 every 8 connection(s)
I20260807 19:31:34.852052 15633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:34.855934 15703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:34.859305 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15633
I20260807 19:31:34.859563 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:34.860308 9058 external_mini_cluster.cc:1428] Setting key f96670c21b5dbdb062fde6cd3508af15
I20260807 19:31:34.869377 15703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Bootstrap starting.
I20260807 19:31:34.873389 15703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:34.874791 15703 log.cc:824] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:31:34 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:34.878199 15703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: No bootstrap required, opened a new log
I20260807 19:31:34.884331 15706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:34.862380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.8.216.190:40249)
0807 19:31:34.863170 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:34.863182 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:34.863229 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:31:34.865025 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:34.865051 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:34.865495 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:34.865843 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:34.867738 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:34.869104 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:34.870817 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:34.871514 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:34.871652 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:34.876601 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:34.876609 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:34.876625 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:34.876663 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:34.879897 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:34.880683 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:34.880690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:34.880696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:34.880889 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:34.881308 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:34.881313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:34.881318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:34.881906 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:34.882273 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:34.882733 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 19:31:34.883199 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":359,"threads_started":1}
I20260807 19:31:34.886449 15703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } }
I20260807 19:31:34.886873 15703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:34.886996 15703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eca52ce8d2d4ce4a2a2f52245054831, State: Initialized, Role: FOLLOWER
I20260807 19:31:34.887604 15703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } }
I20260807 19:31:34.887843 15703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:34.888000 15703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:34.888202 15703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:34.890762 15703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } }
I20260807 19:31:34.891300 15703 leader_election.cc:302] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eca52ce8d2d4ce4a2a2f52245054831; no voters:
I20260807 19:31:34.891918 15703 leader_election.cc:288] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:34.892081 15708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:34.893150 15708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 LEADER]: Becoming Leader. State: Replica: 1eca52ce8d2d4ce4a2a2f52245054831, State: Running, Role: LEADER
I20260807 19:31:34.893760 15708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } }
I20260807 19:31:34.894531 15703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:34.897246 15710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eca52ce8d2d4ce4a2a2f52245054831. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } }
I20260807 19:31:34.897318 15709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } }
I20260807 19:31:34.897702 15710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:34.897766 15709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:34.904942 15716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:34.910635 15716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:34.920190 15716 catalog_manager.cc:1347] Generated new cluster ID: ba522174f53942178942fa14e8246f0d
I20260807 19:31:34.920312 15716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:34.937718 15716 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:34.939105 15716 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:34.948709 15716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Generated new TSK 0
I20260807 19:31:34.949611 15716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
I20260807 19:31:35.019531 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:35.369510 15730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:35.369829 15730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:35.369931 15730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:35.378288 15730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:35.378413 15730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:35.378531 15730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:35.388785 15730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:35.390128 15730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:35.391450 15730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:35.405947 15736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:35.405884 15738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:35.405884 15735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:35.406934 15730 server_base.cc:1034] running on GCE node
I20260807 19:31:35.407567 15730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:35.408614 15730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:35.409852 15730 hybrid_clock.cc:648] HybridClock initialized: now 1786131095409798 us; error 44 us; skew 500 ppm
I20260807 19:31:35.410174 15730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:35 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131095, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:35.413669 15730 init.cc:377] Logged in from keytab as oryx/127.8.216.129@KRBTEST.COM (short username oryx)
I20260807 19:31:35.415293 15730 webserver.cc:460] Webserver started at http://127.8.216.129:35659/ using document root <none> and password file <none>
I20260807 19:31:35.416062 15730 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:35.416254 15730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:35.416579 15730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:35.419595 15730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "d7b2c05ea57242c193e09d98ec58c18a"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "6d1bb28df3f770ed5a894c6f798d0e49"
server_key_iv: "35b2a3a21f8b208190faa1996d4fa27b"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.420468 15730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "d7b2c05ea57242c193e09d98ec58c18a"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "6d1bb28df3f770ed5a894c6f798d0e49"
server_key_iv: "35b2a3a21f8b208190faa1996d4fa27b"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.425880 15730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:35.429479 15745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:35.430871 15730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:35.431106 15730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "d7b2c05ea57242c193e09d98ec58c18a"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "6d1bb28df3f770ed5a894c6f798d0e49"
server_key_iv: "35b2a3a21f8b208190faa1996d4fa27b"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.431370 15730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:35.449301 15730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:35.453675 15730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:35.453995 15730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:35.455022 15730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:35.456524 15730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:35.456629 15730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:35.456780 15730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:35.456853 15730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:35.501335 15730 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:34139
I20260807 19:31:35.501430 15863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:34139 every 8 connection(s)
I20260807 19:31:35.503283 15730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:35.506173 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15730
I20260807 19:31:35.506438 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:35.507232 9058 external_mini_cluster.cc:1428] Setting key 473198a7d9dd5ac770a3664553a72463
Aug 07 19:31:35 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131095, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:31:35.535655 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:35.507840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:35395 (local address 127.8.216.190:40249)
0807 19:31:35.508352 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:35.508359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:35.508703 (+ 344us) server_negotiation.cc:407] Connection header received
0807 19:31:35.510788 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:35.510797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:35.510893 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:35.511024 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:35.518797 (+ 7773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:35.519534 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:35.522464 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:35.523303 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:35.523480 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:35.528874 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:35.528885 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:35.528892 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:35.528963 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:35.531883 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:35.533017 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:35.533021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:35.533024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:35.533163 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:35.533695 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:35.533698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:35.533700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:35.533982 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:35.534096 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:35.535137 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0807 19:31:35.535329 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":281,"threads_started":1}
I20260807 19:31:35.537106 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:35.507217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:35395)
0807 19:31:35.508536 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:35.508588 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:35.510521 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:35.511323 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:35.511339 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:35.512312 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:35.513275 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:35.513309 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:35.519763 (+ 6454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:35.519771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:35.522229 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:35.522253 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:35.523616 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:35.523627 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:35.523999 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:35.524877 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:35.524921 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:35.528517 (+ 3596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:35.532082 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:35.532093 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:35.532116 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:35.532849 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:35.533290 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:35.533296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:35.533301 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:35.533575 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:35.534129 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:35.534141 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:35.534658 (+ 517us) client_negotiation.cc:769] Sending connection context
0807 19:31:35.535032 (+ 374us) client_negotiation.cc:240] Negotiation successful
0807 19:31:35.535520 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":360,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.130@KRBTEST.COM" created.
I20260807 19:31:35.538797 15864 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:35.539134 15864 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:35.539921 15864 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
I20260807 19:31:35.542215 15667 ts_manager.cc:194] Registered new tserver with Master: d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139)
I20260807 19:31:35.543697 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:35395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
I20260807 19:31:35.567636 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:35.912241 15870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:35.912552 15870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:35.912730 15870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:35.920192 15870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:35.920344 15870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:35.920548 15870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:35.930698 15870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:35.932042 15870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:35.933530 15870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:35.947057 15875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:35.947013 15879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:35.947095 15876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:35.948094 15870 server_base.cc:1034] running on GCE node
I20260807 19:31:35.948736 15870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:35.949695 15870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:35.950893 15870 hybrid_clock.cc:648] HybridClock initialized: now 1786131095950832 us; error 61 us; skew 500 ppm
I20260807 19:31:35.951237 15870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:35 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131095, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:35.959022 15870 init.cc:377] Logged in from keytab as oryx/127.8.216.130@KRBTEST.COM (short username oryx)
I20260807 19:31:35.960736 15870 webserver.cc:460] Webserver started at http://127.8.216.130:42485/ using document root <none> and password file <none>
I20260807 19:31:35.961516 15870 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:35.961671 15870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:35.961995 15870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:35.965086 15870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "1f0ecec2bb45417d9f28252354030dc6"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "2d4391561631b4d2ae6c267afdb1f06f"
server_key_iv: "0993aafe9363c81c1550f115e4a7e188"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.965878 15870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "1f0ecec2bb45417d9f28252354030dc6"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "2d4391561631b4d2ae6c267afdb1f06f"
server_key_iv: "0993aafe9363c81c1550f115e4a7e188"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.971242 15870 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:31:35.974980 15886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:35.976424 15870 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:35.976605 15870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "1f0ecec2bb45417d9f28252354030dc6"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "2d4391561631b4d2ae6c267afdb1f06f"
server_key_iv: "0993aafe9363c81c1550f115e4a7e188"
server_key_version: "encryptionkey@0"
I20260807 19:31:35.976825 15870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:36.012450 15870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:36.016896 15870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:36.017231 15870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:36.018292 15870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:36.019918 15870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:36.020040 15870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:36.020226 15870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:36.020308 15870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:36.054276 15870 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:39767
I20260807 19:31:36.054314 16002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:39767 every 8 connection(s)
I20260807 19:31:36.055586 15870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:36.063768 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 15870
I20260807 19:31:36.063975 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:36.064575 9058 external_mini_cluster.cc:1428] Setting key 0769bb7c3c1b9ef884460c50d79bda45
Aug 07 19:31:36 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131095, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:36.077804 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:36.058260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:55363 (local address 127.8.216.190:40249)
0807 19:31:36.058738 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:36.058743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:36.059588 (+ 845us) server_negotiation.cc:407] Connection header received
0807 19:31:36.061529 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:36.061536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:36.061628 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:36.061750 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:36.063259 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.064464 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.066233 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.066803 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.066929 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:36.070525 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:36.070535 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:36.070542 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:36.070591 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:36.074461 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:36.075208 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:36.075216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:36.075222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:36.075392 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:36.075950 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:36.075954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:36.075956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:36.076318 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:36.076434 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:36.077342 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 19:31:36.077520 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":262,"threads_started":1}
I20260807 19:31:36.079485 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:36.058721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:55363)
0807 19:31:36.059412 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:36.059464 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:36.061225 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:36.061995 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:36.062004 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:36.062530 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:36.063078 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:36.063093 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.064623 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.064627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:36.066087 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:36.066100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.067034 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.067038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:36.067236 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:36.067947 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:36.067977 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:36.070262 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:36.074614 (+ 4352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:36.074622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:36.074635 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:36.075078 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:36.075533 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:36.075540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:36.075545 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:36.075807 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:36.076446 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:36.076458 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:36.076965 (+ 507us) client_negotiation.cc:769] Sending connection context
0807 19:31:36.077285 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 19:31:36.077908 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":267,"threads_started":1}
I20260807 19:31:36.081205 16003 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:36.081525 16003 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:36.082352 16003 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
I20260807 19:31:36.083914 15667 ts_manager.cc:194] Registered new tserver with Master: 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:36.084815 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:55363
WARNING: no policy specified for oryx/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
I20260807 19:31:36.123596 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:36.465320 16010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:36.465677 16010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:36.465852 16010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:36.473292 16010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:36.473475 16010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:36.473655 16010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:36.484293 16010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:40249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:36.485687 16010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:36.487035 16010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:36.500823 16017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:36.501003 16015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:36.501190 16019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:36.504881 16010 server_base.cc:1034] running on GCE node
I20260807 19:31:36.505738 16010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:36.506898 16010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:36.508199 16010 hybrid_clock.cc:648] HybridClock initialized: now 1786131096508107 us; error 39 us; skew 500 ppm
I20260807 19:31:36.508586 16010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:36 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131096, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:36.511915 16010 init.cc:377] Logged in from keytab as oryx/127.8.216.131@KRBTEST.COM (short username oryx)
I20260807 19:31:36.513681 16010 webserver.cc:460] Webserver started at http://127.8.216.131:42415/ using document root <none> and password file <none>
I20260807 19:31:36.514475 16010 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:36.514636 16010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:36.514964 16010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:36.518144 16010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "6a6a28a6844f4794a7831fba938ae07e"
format_stamp: "Formatted at 2026-08-07 19:31:36 on dist-test-slave-8w9v"
server_key: "fb24cfa5c959facc9f0936ab881ee355"
server_key_iv: "cf96956428e8dda7e919af366125be1e"
server_key_version: "encryptionkey@0"
I20260807 19:31:36.518973 16010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "6a6a28a6844f4794a7831fba938ae07e"
format_stamp: "Formatted at 2026-08-07 19:31:36 on dist-test-slave-8w9v"
server_key: "fb24cfa5c959facc9f0936ab881ee355"
server_key_iv: "cf96956428e8dda7e919af366125be1e"
server_key_version: "encryptionkey@0"
I20260807 19:31:36.524479 16010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:31:36.527987 16026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:36.529368 16010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:36.529563 16010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "6a6a28a6844f4794a7831fba938ae07e"
format_stamp: "Formatted at 2026-08-07 19:31:36 on dist-test-slave-8w9v"
server_key: "fb24cfa5c959facc9f0936ab881ee355"
server_key_iv: "cf96956428e8dda7e919af366125be1e"
server_key_version: "encryptionkey@0"
I20260807 19:31:36.529769 16010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:36.546706 15864 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
I20260807 19:31:36.550380 16010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:36.555109 16010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:36.555433 16010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:36.556550 16010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:36.557963 16010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:36.558072 16010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:36.558225 16010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:36.558300 16010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:36.593295 16010 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:39819
I20260807 19:31:36.593345 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:39819 every 8 connection(s)
I20260807 19:31:36.594661 16010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:36.598827 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16010
I20260807 19:31:36.599061 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:36.599678 9058 external_mini_cluster.cc:1428] Setting key d10ee58fe373d0e6b5231c81a234c97f
Aug 07 19:31:36 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131096, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:36.616926 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:36.597511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54243 (local address 127.8.216.190:40249)
0807 19:31:36.598027 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:36.598032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:36.599014 (+ 982us) server_negotiation.cc:407] Connection header received
0807 19:31:36.601453 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:36.601460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:36.601569 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:36.601683 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:36.603499 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.604497 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.606160 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.606560 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.606660 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:36.610314 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:36.610319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:36.610322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:36.610357 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:36.613197 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:36.614173 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:36.614179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:36.614183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:36.614361 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:36.614920 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:36.614924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:36.614928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:36.615318 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:36.615489 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:36.616415 (+ 926us) server_negotiation.cc:299] Negotiation successful
0807 19:31:36.616608 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":268,"threads_started":1}
I20260807 19:31:36.617928 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:36.597970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:54243)
0807 19:31:36.598813 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:36.598869 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:36.601111 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:36.602071 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:36.602080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:36.602627 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:36.603327 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:36.603346 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.604661 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.604669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:36.606025 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:36.606038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:36.606754 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:36.606759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:36.606980 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:36.607734 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:36.607758 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:36.610105 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:36.613357 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:36.613368 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:36.613387 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:36.614037 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:36.614515 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:36.614520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:36.614524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:36.614792 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:36.615454 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:36.615461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:36.616003 (+ 542us) client_negotiation.cc:769] Sending connection context
0807 19:31:36.616326 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 19:31:36.616779 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":281,"threads_started":1}
I20260807 19:31:36.619288 16147 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:36.619633 16147 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:36.620471 16147 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
I20260807 19:31:36.622040 15667 ts_manager.cc:194] Registered new tserver with Master: 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819)
I20260807 19:31:36.622874 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:54243
I20260807 19:31:36.628264 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.216.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.189.keytab.
Entry for principal oryx/127.8.216.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.189.keytab.
I20260807 19:31:36.687309 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:42159
--webserver_interface=127.8.216.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.189.keytab
--principal=oryx/127.8.216.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:40249,127.8.216.189:42159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:37.032775 16153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:37.033190 16153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:37.033354 16153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:37.041234 16153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:37.041406 16153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:37.041519 16153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:37.041616 16153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:37.041702 16153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:37.051977 16153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:40249,127.8.216.189:42159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.189.keytab
--principal=oryx/127.8.216.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:42159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.216.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
W20260807 19:31:37.052407 16153 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.216.190:40249,127.8.216.189:42159'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 19:31:37.053547 16153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:37.055389 16153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:37.068825 16159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:37.068894 16160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:37.069655 16162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:37.070477 16153 server_base.cc:1034] running on GCE node
I20260807 19:31:37.071353 16153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:37.072902 16153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:37.074124 16153 hybrid_clock.cc:648] HybridClock initialized: now 1786131097074070 us; error 27 us; skew 500 ppm
I20260807 19:31:37.074417 16153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131097, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:37.077807 16153 init.cc:377] Logged in from keytab as oryx/127.8.216.189@KRBTEST.COM (short username oryx)
I20260807 19:31:37.079214 16153 webserver.cc:460] Webserver started at http://127.8.216.189:38599/ using document root <none> and password file <none>
I20260807 19:31:37.079943 16153 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:37.080113 16153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:37.080480 16153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:37.083578 16153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/instance:
uuid: "bd2f295b92284022a3164d79b48f824a"
format_stamp: "Formatted at 2026-08-07 19:31:37 on dist-test-slave-8w9v"
server_key: "29d0601d3ae1263e5e86192083d32c44"
server_key_iv: "56be398dceded21f3d8f607a6347aecb"
server_key_version: "encryptionkey@0"
I20260807 19:31:37.084523 16153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal/instance:
uuid: "bd2f295b92284022a3164d79b48f824a"
format_stamp: "Formatted at 2026-08-07 19:31:37 on dist-test-slave-8w9v"
server_key: "29d0601d3ae1263e5e86192083d32c44"
server_key_iv: "56be398dceded21f3d8f607a6347aecb"
server_key_version: "encryptionkey@0"
I20260807 19:31:37.087827 16003 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
I20260807 19:31:37.090073 16153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:31:37.093735 16170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:37.095204 16153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:31:37.095393 16153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "bd2f295b92284022a3164d79b48f824a"
format_stamp: "Formatted at 2026-08-07 19:31:37 on dist-test-slave-8w9v"
server_key: "29d0601d3ae1263e5e86192083d32c44"
server_key_iv: "56be398dceded21f3d8f607a6347aecb"
server_key_version: "encryptionkey@0"
I20260807 19:31:37.095619 16153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:37.110832 16153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:37.115096 16153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:37.115382 16153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:37.133888 16153 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.189:42159
I20260807 19:31:37.133858 16221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.189:42159 every 8 connection(s)
I20260807 19:31:37.135322 16153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
I20260807 19:31:37.139230 16222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:37.142406 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16153
I20260807 19:31:37.147008 16222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131097, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.189@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:37.167704 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.150564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.8.216.190:40249)
0807 19:31:37.151149 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.151156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.151652 (+ 496us) server_negotiation.cc:407] Connection header received
0807 19:31:37.151879 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.151885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.151962 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.152082 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.153584 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.154672 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.156836 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.157259 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.157361 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.161254 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.161259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.161262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.161301 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.164803 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.165632 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.165636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.165640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.165789 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.166246 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.166251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.166254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.166660 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.166802 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.167285 (+ 483us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.167476 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":288,"threads_started":1}
I20260807 19:31:37.168246 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.150642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38762)
0807 19:31:37.151500 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.151546 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.151735 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.152163 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.152170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.153084 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.153407 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.153420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.154823 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.154827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.156672 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.156688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.157505 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.157515 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.157735 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.158402 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.158437 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.161023 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.164921 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.164930 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.164946 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.165496 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.165902 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.165905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.165908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.166112 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.166786 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.166792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.166977 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.167182 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.167493 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":435,"threads_started":1}
I20260807 19:31:37.168268 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.144696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.8.216.189:42159)
0807 19:31:37.145499 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.145515 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.145566 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:31:37.147726 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.147758 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.148331 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.148789 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.150189 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.152223 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.153948 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.154972 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.155111 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.160260 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.160271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.160290 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.160332 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.163173 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.164027 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.164033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.164040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.164262 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.164713 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.164717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.164719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.165481 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.165742 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.166279 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.166736 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":326,"threads_started":1}
I20260807 19:31:37.170142 16222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131097, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.189@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
I20260807 19:31:37.182997 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.171152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53712)
0807 19:31:37.171348 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.171368 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.171503 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.171790 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.171794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.172241 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.172371 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.172379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.173328 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.173334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.174181 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.174192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.174869 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.174874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.175096 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.175634 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.175671 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.177822 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.180685 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.180692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.180696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.181091 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.181428 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.181434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.181438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.181582 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.182106 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.182111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.182258 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.182392 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.182722 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":4}
I20260807 19:31:37.183259 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.171151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.8.216.189:42159)
0807 19:31:37.171347 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.171353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.171454 (+ 101us) server_negotiation.cc:407] Connection header received
0807 19:31:37.171614 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.171620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.171680 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.171777 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.172501 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.173188 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.174319 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.174725 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.174861 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.178009 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.178014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.178017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.178044 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.180557 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.181204 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.181211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.181214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.181325 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.181681 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.181689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.181693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.181968 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.182120 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.182492 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.182890 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":228,"server-negotiator.queue_time_us":70}
I20260807 19:31:37.187983 16222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Bootstrap starting.
I20260807 19:31:37.191121 16222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:37.192277 16222 log.cc:824] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:37.194979 16222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: No bootstrap required, opened a new log
I20260807 19:31:37.202199 16222 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } }
I20260807 19:31:37.202556 16222 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:37.202654 16222 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2f295b92284022a3164d79b48f824a, State: Initialized, Role: FOLLOWER
I20260807 19:31:37.203254 16222 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } }
I20260807 19:31:37.204209 16229 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } } }
I20260807 19:31:37.204560 16229 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:37.205291 16222 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:31:37.214083 16240 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:31:37.214246 16240 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:31:37.217099 16153 master_runner.cc:428] Detected that this master bd2f295b92284022a3164d79b48f824a is joining an existing cluster
I20260807 19:31:37.217196 16153 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.216.189:42159
I20260807 19:31:37.244683 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.221179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.8.216.189:42159)
0807 19:31:37.221412 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.221420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.227476 (+ 6056us) server_negotiation.cc:407] Connection header received
0807 19:31:37.228197 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.228206 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.228300 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.228385 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.230526 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.231759 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.233927 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.234774 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.234872 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.237479 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.237486 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.237493 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.237534 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.241476 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.242282 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.242289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.242294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.242454 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.242879 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.242883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.242887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.243269 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.243450 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.243960 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.244241 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":65}
I20260807 19:31:37.245769 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.221583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53716)
0807 19:31:37.227339 (+ 5756us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.227372 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.227987 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.228643 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.228651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.229626 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.230322 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.230346 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.231994 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.231998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.233772 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.233785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.235219 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.235224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.235413 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.236205 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.236232 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.237298 (+ 1066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.241619 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.241626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.241633 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.242154 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.242598 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.242602 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.242606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.242758 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.243524 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.243531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.243694 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.244984 (+ 1290us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.245309 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5539,"mutex_wait_us":18,"thread_start_us":8594,"threads_started":1}
I20260807 19:31:37.246809 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.221210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.8.216.190:40249)
0807 19:31:37.221427 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.221433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.230636 (+ 9203us) server_negotiation.cc:407] Connection header received
0807 19:31:37.230911 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.230919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.231039 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.231117 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.232674 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.233764 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.235871 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.236637 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.236778 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.239934 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.239954 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.239961 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.240010 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.243682 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.244511 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.244516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.244519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.244684 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.245167 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.245173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.245178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.245546 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.245703 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.246345 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.246584 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:31:37.247756 16247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.221903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38764)
0807 19:31:37.230508 (+ 8605us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.230529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.230753 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.231323 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.231328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.232074 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.232469 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.232481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.234033 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.234038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.235710 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.235726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.237520 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.237526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.237718 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.238407 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.238438 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.239656 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.243837 (+ 4181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.243844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.243850 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.244361 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.244853 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.244858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.244861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.245035 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.245798 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.245803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.246020 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.247033 (+ 1013us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.247301 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8459,"mutex_wait_us":4,"thread_start_us":5592,"threads_started":1}
WARNING: no policy specified for oryx/127.8.216.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.188@KRBTEST.COM" created.
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.190@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
I20260807 19:31:37.271883 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.253988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.8.216.189:42159)
0807 19:31:37.254178 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.254184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.254555 (+ 371us) server_negotiation.cc:407] Connection header received
0807 19:31:37.254837 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.254845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.255023 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.255159 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.256399 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.257650 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.259851 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.260631 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.260768 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.265476 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.265485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.265492 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.265547 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.268861 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.269669 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.269675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.269678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.269822 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.270266 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.270272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.270275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.270568 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.270736 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.271228 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.271480 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:31:37.271924 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.253683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53718)
0807 19:31:37.254384 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.254423 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.254619 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.255230 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.255236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.255872 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.256202 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.256216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.257822 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.257829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.259654 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.259674 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.260828 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.260837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.261087 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.261917 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.261944 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.265209 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.269051 (+ 3842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.269067 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.269075 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.269532 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.269957 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.269962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.269966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.270147 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.270731 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.270740 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.270925 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.271131 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.271500 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":513,"threads_started":1}
I20260807 19:31:37.274258 15667 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.216.189:42159: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bd2f295b92284022a3164d79b48f824a"
member_type: NON_VOTER
last_known_addr {
host: "127.8.216.189"
port: 42159
}
attrs {
promote: true
}
}
dest_uuid: "1eca52ce8d2d4ce4a2a2f52245054831"
cas_config_opid_index: -1
I20260807 19:31:37.275609 15667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } }
I20260807 19:31:37.284403 16197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:37.284080 16255 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bd2f295b92284022a3164d79b48f824a (127.8.216.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } } }
I20260807 19:31:37.288314 16255 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.216.189:42159
I20260807 19:31:37.288487 16197 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 19:31:37.289582 16153 master.cc:537] Master@127.8.216.189:42159 shutting down...
I20260807 19:31:37.291932 16254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } } }
I20260807 19:31:37.292482 16254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:37.293845 16255 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.188.keytab.
Entry for principal oryx/127.8.216.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.188.keytab.
I20260807 19:31:37.304235 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:34395
--webserver_interface=127.8.216.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.188.keytab
--principal=oryx/127.8.216.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:37.313825 16153 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:37.314576 16153 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:37.314937 16153 tablet_replica.cc:330] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: stopping tablet replica
W20260807 19:31:37.319041 15654 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 19:31:37.324937 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.324418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53724)
0807 19:31:37.324593 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.324716 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:42159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:31:37.325274 15654 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 -> Peer bd2f295b92284022a3164d79b48f824a (127.8.216.189:42159): Couldn't send request to peer bd2f295b92284022a3164d79b48f824a. Status: Network error: Client connection negotiation failed: client connection to 127.8.216.189:42159: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:31:37.327035 16153 master.cc:552] Master@127.8.216.189:42159 shutdown complete.
I20260807 19:31:37.327171 16153 master_runner.cc:305] Clearing existing system tablet
I20260807 19:31:37.327725 16153 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:37.327831 16153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:37.332858 16153 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:31:37.340821 16261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:37.341300 16153 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:37.341645 16153 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.006s
I20260807 19:31:37.341751 16153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "bd2f295b92284022a3164d79b48f824a"
format_stamp: "Formatted at 2026-08-07 19:31:37 on dist-test-slave-8w9v"
server_key: "29d0601d3ae1263e5e86192083d32c44"
server_key_iv: "56be398dceded21f3d8f607a6347aecb"
server_key_version: "encryptionkey@0"
I20260807 19:31:37.347966 16153 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:37.355243 16153 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:31:37.355449 16153 log.cc:1196] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 19:31:37.355890 16153 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Deleting consensus metadata
I20260807 19:31:37.356243 16153 master_runner.cc:315] Copying system tablet from 127.8.216.190:40249
I20260807 19:31:37.358911 16153 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: tablet copy: Beginning tablet copy session from remote peer at address 127.8.216.190:40249
I20260807 19:31:37.375083 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.359820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.8.216.190:40249)
0807 19:31:37.360010 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.360016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.360582 (+ 566us) server_negotiation.cc:407] Connection header received
0807 19:31:37.360778 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.360784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.360884 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.361000 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:37.362103 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.363289 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.365051 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.365751 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.365910 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.368250 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:37.368257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:37.368262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:37.368304 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:37.371920 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.372799 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.372805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.372809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.372983 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:37.373428 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:37.373434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:37.373437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:37.373803 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:37.373979 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.374442 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.374686 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:31:37.375133 16267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.359739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38774)
0807 19:31:37.360436 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.360469 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.360625 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.361046 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.361051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.361608 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:37.361893 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.361908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.363460 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.363465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.364883 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.364903 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.365936 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.365942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.366126 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.366891 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:37.366918 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:37.367995 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:37.372076 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.372082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.372086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.372660 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.373112 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:37.373116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:37.373120 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:37.373298 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:37.373955 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:37.373960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:37.374148 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.374306 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.374663 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":400,"threads_started":1}
I20260807 19:31:37.376510 15688 tablet_copy_service.cc:140] P 1eca52ce8d2d4ce4a2a2f52245054831: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bd2f295b92284022a3164d79b48f824a ({username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:38774)
I20260807 19:31:37.376843 15688 tablet_copy_service.cc:161] P 1eca52ce8d2d4ce4a2a2f52245054831: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bd2f295b92284022a3164d79b48f824a at {username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:38774: session id = bd2f295b92284022a3164d79b48f824a-00000000000000000000000000000000
I20260807 19:31:37.381651 15688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:31:37.383996 16153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:31:37.391080 16153 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: tablet copy: Starting download of 0 data blocks...
I20260807 19:31:37.391417 16153 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: tablet copy: Starting download of 1 WAL segments...
I20260807 19:31:37.394480 16153 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:31:37.399185 15688 tablet_copy_service.cc:342] P 1eca52ce8d2d4ce4a2a2f52245054831: Request end of tablet copy session bd2f295b92284022a3164d79b48f824a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:38774
I20260807 19:31:37.399421 15688 tablet_copy_service.cc:433] P 1eca52ce8d2d4ce4a2a2f52245054831: ending tablet copy session bd2f295b92284022a3164d79b48f824a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bd2f295b92284022a3164d79b48f824a
I20260807 19:31:37.412563 16153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:37.415431 16268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:37.425454 16269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:37.426219 16271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:37.426728 16153 server_base.cc:1034] running on GCE node
I20260807 19:31:37.427158 16153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:37.427903 16153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:37.429147 16153 hybrid_clock.cc:648] HybridClock initialized: now 1786131097429060 us; error 86 us; skew 500 ppm
I20260807 19:31:37.429376 16153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:37 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131097, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:37.431653 16153 init.cc:377] Logged in from keytab as oryx/127.8.216.189@KRBTEST.COM (short username oryx)
I20260807 19:31:37.436448 16153 webserver.cc:460] Webserver started at http://127.8.216.189:40415/ using document root <none> and password file <none>
I20260807 19:31:37.436838 16153 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:37.436930 16153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:37.440161 16153 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 19:31:37.442157 16279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:37.442570 16153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260807 19:31:37.442761 16153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "bd2f295b92284022a3164d79b48f824a"
format_stamp: "Formatted at 2026-08-07 19:31:37 on dist-test-slave-8w9v"
server_key: "29d0601d3ae1263e5e86192083d32c44"
server_key_iv: "56be398dceded21f3d8f607a6347aecb"
server_key_version: "encryptionkey@0"
I20260807 19:31:37.443020 16153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:37.461727 16153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:37.466395 16153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:37.506800 16153 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.189:42159
I20260807 19:31:37.506834 16332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.189:42159 every 8 connection(s)
I20260807 19:31:37.508097 16153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
I20260807 19:31:37.513988 16333 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:31:37.518121 16333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Bootstrap starting.
I20260807 19:31:37.537513 16333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:31:37.538065 16333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Bootstrap complete.
I20260807 19:31:37.539199 16333 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } }
I20260807 19:31:37.539515 16333 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bd2f295b92284022a3164d79b48f824a, State: Initialized, Role: LEARNER
I20260807 19:31:37.539769 16333 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } }
I20260807 19:31:37.540922 16333 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:37.540678 16336 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } } }
I20260807 19:31:37.541188 16336 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: LEARNER
I20260807 19:31:37.554059 16347 catalog_manager.cc:1259] Loaded cluster ID: ba522174f53942178942fa14e8246f0d
I20260807 19:31:37.554194 16347 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: loading cluster ID for follower catalog manager: success
I20260807 19:31:37.556854 16347 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: acquiring CA information for follower catalog manager: success
I20260807 19:31:37.558811 16347 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:31:37.625923 16147 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
I20260807 19:31:37.757074 16251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.739059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53730)
0807 19:31:37.739253 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:37.739282 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:37.739404 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:37.741451 (+ 2047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:37.741459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:37.741505 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:37.741942 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.741956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.743820 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.743825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.752749 (+ 8924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:37.752771 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.755567 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.755575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:37.756040 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.756145 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 19:31:37.756231 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 19:31:37.756465 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:31:37.758910 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:37.739798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.8.216.189:42159)
0807 19:31:37.741034 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:37.741039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:37.741056 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:31:37.741123 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:37.741127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:37.741259 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:37.741400 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:37.742130 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.743594 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.752982 (+ 9388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:37.755269 (+ 2287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:37.757448 (+ 2179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:37.757881 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:37.757954 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 19:31:37.758162 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":744,"threads_started":1}
I20260807 19:31:37.760847 16308 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 19:31:37.761552 16336 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eca52ce8d2d4ce4a2a2f52245054831. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } } }
I20260807 19:31:37.761859 16336 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: LEARNER
W20260807 19:31:37.965653 16258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:37.966078 16258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:37.966320 16258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:37.978046 16258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:37.978366 16258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:37.978580 16258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:37.978775 16258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:37.978964 16258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:37.995277 16258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.188.keytab
--principal=oryx/127.8.216.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.216.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:37.997206 16258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:37.998998 16258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:38.020211 16354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:38.020788 16355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:38.021243 16357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:38.022575 16258 server_base.cc:1034] running on GCE node
I20260807 19:31:38.023880 16258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:38.025868 16258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:38.027194 16258 hybrid_clock.cc:648] HybridClock initialized: now 1786131098027053 us; error 45 us; skew 500 ppm
I20260807 19:31:38.027781 16258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131098, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:38.033030 16258 init.cc:377] Logged in from keytab as oryx/127.8.216.188@KRBTEST.COM (short username oryx)
I20260807 19:31:38.035598 16258 webserver.cc:460] Webserver started at http://127.8.216.188:37305/ using document root <none> and password file <none>
I20260807 19:31:38.036782 16258 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:38.037060 16258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:38.037515 16258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:38.041996 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/instance:
uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
format_stamp: "Formatted at 2026-08-07 19:31:38 on dist-test-slave-8w9v"
server_key: "2c0342479437f4114077f60ba3830ba9"
server_key_iv: "d2fb8348db5830c1b635b44fbae0e10f"
server_key_version: "encryptionkey@0"
I20260807 19:31:38.042991 16258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal/instance:
uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
format_stamp: "Formatted at 2026-08-07 19:31:38 on dist-test-slave-8w9v"
server_key: "2c0342479437f4114077f60ba3830ba9"
server_key_iv: "d2fb8348db5830c1b635b44fbae0e10f"
server_key_version: "encryptionkey@0"
I20260807 19:31:38.050565 16258 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 19:31:38.056221 16364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:38.057770 16258 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 19:31:38.058048 16258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
format_stamp: "Formatted at 2026-08-07 19:31:38 on dist-test-slave-8w9v"
server_key: "2c0342479437f4114077f60ba3830ba9"
server_key_iv: "d2fb8348db5830c1b635b44fbae0e10f"
server_key_version: "encryptionkey@0"
I20260807 19:31:38.058399 16258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:38.113915 16258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:38.121192 16258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:38.121795 16258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:38.172377 16258 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.188:34395
I20260807 19:31:38.172598 16416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.188:34395 every 8 connection(s)
I20260807 19:31:38.174258 16258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
I20260807 19:31:38.180174 16417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:38.181310 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16258
I20260807 19:31:38.194511 16417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131098, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.188@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:38.220433 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.184627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56800 (local address 127.8.216.188:34395)
0807 19:31:38.186537 (+ 1910us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.186549 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.186620 (+ 71us) server_negotiation.cc:407] Connection header received
0807 19:31:38.190704 (+ 4084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.190732 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.191233 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.191783 (+ 550us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.194320 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.196556 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.199205 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.200282 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.200409 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.208009 (+ 7600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.208022 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.208036 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.208078 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.214070 (+ 5992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.214963 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.214970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.214981 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.215171 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.215826 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.215830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.215832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.216675 (+ 843us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.217044 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.217769 (+ 725us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.218360 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1603,"thread_start_us":1159,"threads_started":1}
I20260807 19:31:38.227023 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.198311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38788)
0807 19:31:38.202725 (+ 4414us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.202776 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.203022 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.203530 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.203540 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.204896 (+ 1356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.205795 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.205814 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.207839 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.207845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.210330 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.210348 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.211987 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.211994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.212320 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.213254 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.213315 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.217379 (+ 4064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.223447 (+ 6068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.223456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.223462 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.224444 (+ 982us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.224924 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.224927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.224930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.225110 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.225760 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.225765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.225987 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.226197 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.226451 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4062,"thread_start_us":3771,"threads_started":1}
I20260807 19:31:38.227095 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.198231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.8.216.190:40249)
0807 19:31:38.199340 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.199347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.202895 (+ 3548us) server_negotiation.cc:407] Connection header received
0807 19:31:38.203187 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.203192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.203271 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.203420 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.206034 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.207474 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.210513 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.211602 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.211789 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.217676 (+ 5887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.217684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.217689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.217731 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.223302 (+ 5571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.224591 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.224595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.224597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.224821 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.225211 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.225215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.225217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.225638 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.225765 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.226308 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.226591 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":656,"threads_started":1}
I20260807 19:31:38.228757 16417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131098, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.188@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
I20260807 19:31:38.247547 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.229520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53742)
0807 19:31:38.229763 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.229780 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.229869 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.230278 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.230282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.230923 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.231205 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.231214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.232570 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.232574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.234139 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.234152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.235332 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.235337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.235592 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.236640 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.236659 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.239734 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.244713 (+ 4979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.244718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.244721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.245345 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.245779 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.245782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.245789 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.246027 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.246646 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.246649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.246793 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.246906 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.247239 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 19:31:38.247658 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.229824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.8.216.189:42159)
0807 19:31:38.230033 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.230037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.230053 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:38.230102 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.230105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.230164 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.230242 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.231355 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.232440 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.234327 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.235158 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.235266 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.240030 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.240036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.240039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.240080 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.244533 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.245454 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.245458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.245460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.245668 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.246123 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.246126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.246128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.246544 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.246660 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.246992 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.247233 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:31:38.248975 16417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131098, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.188@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
I20260807 19:31:38.266978 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.250142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.8.216.188:34395)
0807 19:31:38.250278 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.250282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.250294 (+ 12us) server_negotiation.cc:407] Connection header received
0807 19:31:38.250329 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.250332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.250401 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.250504 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.251505 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.252589 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.254117 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.254751 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.254921 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.259479 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.259484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.259487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.259518 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.263580 (+ 4062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.264595 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.264599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.264602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.264801 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.265226 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.265229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.265231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.265656 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.265770 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.266273 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.266581 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":50}
I20260807 19:31:38.266980 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.249792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.0.0.1:56806)
0807 19:31:38.249971 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.249989 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.250068 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.250520 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.250523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.251179 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.251371 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.251380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.252711 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.252719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.253977 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.253990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.254980 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.254986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.255366 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.256108 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.256180 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.259264 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.263703 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.263707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.263710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.264489 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.264905 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.264908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.264911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.265132 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.265780 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.265797 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.266023 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.266180 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.266397 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:31:38.274313 16417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Bootstrap starting.
I20260807 19:31:38.278292 16417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:38.280673 16417 log.cc:824] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:38.283890 16417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: No bootstrap required, opened a new log
I20260807 19:31:38.293313 16417 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } }
I20260807 19:31:38.293767 16417 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:38.293915 16417 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd37ceed4b604db5bbe24d6f070ae26d, State: Initialized, Role: FOLLOWER
I20260807 19:31:38.294772 16417 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } }
I20260807 19:31:38.297489 16426 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } } }
I20260807 19:31:38.298666 16426 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:38.299734 16417 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:38.301232 16424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: NON_VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: true } } }
I20260807 19:31:38.301789 16424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:38.303125 16428 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: attempting to promote NON_VOTER bd2f295b92284022a3164d79b48f824a to VOTER
I20260807 19:31:38.304898 16428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } }
I20260807 19:31:38.310518 16308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 LEARNER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:31:38.311789 16424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:31:38.317984 16428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bd2f295b92284022a3164d79b48f824a (127.8.216.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } }
I20260807 19:31:38.318931 16308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, bd2f295b92284022a3164d79b48f824a (127.8.216.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } }
I20260807 19:31:38.324304 16424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } }
I20260807 19:31:38.324750 16424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:38.325069 16427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } }
I20260807 19:31:38.325806 16427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:38.329694 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15730
W20260807 19:31:38.331459 16444 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:31:38.331790 16444 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:31:38.334422 16446 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } }
I20260807 19:31:38.335044 16446 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:38.339744 16258 master_runner.cc:428] Detected that this master cd37ceed4b604db5bbe24d6f070ae26d is joining an existing cluster
I20260807 19:31:38.340070 16258 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.216.188:34395
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.129@KRBTEST.COM
I20260807 19:31:38.371263 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.331507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.8.216.129:34139)
0807 19:31:38.336672 (+ 5165us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.336682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.336857 (+ 175us) server_negotiation.cc:407] Connection header received
0807 19:31:38.336953 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.336962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.337208 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.337391 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.340579 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.342800 (+ 2221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.344949 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.345723 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.345948 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.353892 (+ 7944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.353927 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.353958 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.354121 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.362805 (+ 8684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.364607 (+ 1802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.364623 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.364630 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.365018 (+ 388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.365784 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.365793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.365800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.366578 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.369364 (+ 2786us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.369632 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.370067 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4895,"thread_start_us":4629,"threads_started":1}
I20260807 19:31:38.374266 15859 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 19:31:38.375947 16370 tablet.cc:2369] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:38.404372 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.358617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.8.216.190:40249)
0807 19:31:38.359140 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.359163 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.368303 (+ 9140us) server_negotiation.cc:407] Connection header received
0807 19:31:38.369283 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.369291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.369451 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.369537 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.372493 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.374376 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.379231 (+ 4855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.380533 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.380640 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.388223 (+ 7583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.388233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.388238 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.388287 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.393969 (+ 5682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.397371 (+ 3402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.397379 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.397382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.399811 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.400774 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.400780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.400783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.401467 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.401759 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.403366 (+ 1607us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.403767 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260807 19:31:38.406909 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.359255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38794)
0807 19:31:38.368160 (+ 8905us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.368184 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.369084 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.369832 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.369837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.371254 (+ 1417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.372273 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.372292 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.374738 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.374743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.379022 (+ 4279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.379045 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.381154 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.381165 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.381536 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.384841 (+ 3305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.384871 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.387579 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.394451 (+ 6872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.394458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.394463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.397145 (+ 2682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.400188 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.400193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.400196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.400600 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.401877 (+ 1277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.401887 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.402953 (+ 1066us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.404664 (+ 1711us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.405787 (+ 1123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8625,"mutex_wait_us":51,"thread_start_us":19211,"threads_started":1}
I20260807 19:31:38.425908 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.358807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.8.216.189:42159)
0807 19:31:38.359067 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.359074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.382562 (+ 23488us) server_negotiation.cc:407] Connection header received
0807 19:31:38.382843 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.382851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.382968 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.383091 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.387787 (+ 4696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.389584 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.394885 (+ 5301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.396270 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.396383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.403969 (+ 7586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.403978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.403983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.404033 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.410422 (+ 6389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.412042 (+ 1620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.412050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.412054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.412919 (+ 865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.422394 (+ 9475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.422402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.422406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.422897 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.423078 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.424772 (+ 1694us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.425176 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:31:38.428316 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.359962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53750)
0807 19:31:38.382420 (+ 22458us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.382449 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.382627 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.383344 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.383349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.386364 (+ 3015us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.387557 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.387572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.391336 (+ 3764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.391355 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.394644 (+ 3289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.394674 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.397911 (+ 3237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.397931 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.398222 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.399387 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.399412 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.403657 (+ 4245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.410613 (+ 6956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.410638 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.410642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.411383 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.421471 (+ 10088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.421477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.421482 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.421752 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.423205 (+ 1453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.423210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.423579 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.426539 (+ 2960us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.427061 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22267,"thread_start_us":8434,"threads_started":1}
I20260807 19:31:38.431484 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.379964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.0.0.1:56810)
0807 19:31:38.383185 (+ 3221us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.383204 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.383390 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.384203 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.384209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.395410 (+ 11201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.397496 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.397514 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.400067 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.400075 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.405579 (+ 5504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.405595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.408948 (+ 3353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.408956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.409247 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.410726 (+ 1479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.410761 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.412912 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.425926 (+ 13014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.425935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.425942 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.426804 (+ 862us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.427426 (+ 622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.427431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.427435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.427694 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.429148 (+ 1454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.429153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.429432 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.429594 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.430419 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3006,"thread_start_us":1172,"threads_started":1}
I20260807 19:31:38.431916 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.379326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.8.216.188:34395)
0807 19:31:38.379655 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.379664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.383274 (+ 3610us) server_negotiation.cc:407] Connection header received
0807 19:31:38.383615 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.383622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.383717 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.383832 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.397650 (+ 13818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.399456 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.406597 (+ 7141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.408520 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.408643 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.413208 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.413216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.413220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.413282 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.419885 (+ 6603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.426979 (+ 7094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.427002 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.427007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.427283 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.427854 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.427858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.427861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.428854 (+ 993us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.429072 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.430043 (+ 971us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.431139 (+ 1096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":789,"server-negotiator.queue_time_us":88}
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.190@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
I20260807 19:31:38.449930 16426 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:38.451381 16426 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } }
I20260807 19:31:38.459959 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.415602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56826 (local address 127.8.216.188:34395)
0807 19:31:38.432227 (+ 16625us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.432280 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.432296 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:38.432379 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.432383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.432548 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.432632 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.435240 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.436917 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.439982 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.441544 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.441698 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.448554 (+ 6856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.448562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.448569 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.448618 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.455539 (+ 6921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.456938 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.456944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.456948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.457151 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.457791 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.457797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.457800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.458331 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.458470 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.459092 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.459402 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16480,"thread_start_us":17661,"threads_started":1}
I20260807 19:31:38.460083 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.415077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.0.0.1:56826)
0807 19:31:38.425443 (+ 10366us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.425471 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.425686 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.432738 (+ 7052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.432745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.434056 (+ 1311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.434969 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.434989 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.437122 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.437128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.439651 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.439671 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.441734 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.441741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.441960 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.443259 (+ 1299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.443343 (+ 84us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.448217 (+ 4874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.455770 (+ 7553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.455778 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.455782 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.456761 (+ 979us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.457312 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.457318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.457322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.457633 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.458498 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.458504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.458746 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.458952 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.459328 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10049,"thread_start_us":11172,"threads_started":1}
I20260807 19:31:38.462613 15667 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.216.188:34395: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
member_type: NON_VOTER
last_known_addr {
host: "127.8.216.188"
port: 34395
}
attrs {
promote: true
}
}
dest_uuid: "1eca52ce8d2d4ce4a2a2f52245054831"
cas_config_opid_index: 6
I20260807 19:31:38.463685 15667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } }
I20260807 19:31:38.466801 16308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 19:31:38.467869 16424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:31:38.471619 16427 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.472352 16308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.478411 16446 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.478922 16446 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:38.479640 16427 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.216.188:34395
I20260807 19:31:38.480485 16428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.481221 16428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:38.481276 16426 leader_election.cc:288] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249), bd2f295b92284022a3164d79b48f824a (127.8.216.189:42159)
I20260807 19:31:38.481647 16258 master.cc:537] Master@127.8.216.188:34395 shutting down...
I20260807 19:31:38.481837 16308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd2f295b92284022a3164d79b48f824a" is_pre_election: true
I20260807 19:31:38.482569 15678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" is_pre_election: true
I20260807 19:31:38.483639 16392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:38.484745 16427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.485129 16427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:38.485792 16368 leader_election.cc:302] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd37ceed4b604db5bbe24d6f070ae26d; no voters: 1eca52ce8d2d4ce4a2a2f52245054831, bd2f295b92284022a3164d79b48f824a
I20260807 19:31:38.490623 16392 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 19:31:38.491941 16427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:38.492166 16426 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:31:38.502543 16258 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:38.503101 16258 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:38.503419 16258 tablet_replica.cc:330] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: stopping tablet replica
W20260807 19:31:38.510842 15653 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 -> Peer cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188:34395): Couldn't send request to peer cd37ceed4b604db5bbe24d6f070ae26d. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:31:38.521512 16258 master.cc:552] Master@127.8.216.188:34395 shutdown complete.
I20260807 19:31:38.521611 16258 master_runner.cc:305] Clearing existing system tablet
I20260807 19:31:38.522068 16258 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:38.522068 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15730
I20260807 19:31:38.522199 16258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:38.541090 16258 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.008s sys 0.012s
I20260807 19:31:38.543783 16460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:38.544281 16258 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:38.544736 16258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 19:31:38.544855 16258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
format_stamp: "Formatted at 2026-08-07 19:31:38 on dist-test-slave-8w9v"
server_key: "2c0342479437f4114077f60ba3830ba9"
server_key_iv: "d2fb8348db5830c1b635b44fbae0e10f"
server_key_version: "encryptionkey@0"
I20260807 19:31:38.551646 16258 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:38.555889 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:34139
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:40249
--webserver_port=35659
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:38.560204 16258 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:31:38.560385 16258 log.cc:1196] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 19:31:38.560958 16258 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Deleting consensus metadata
I20260807 19:31:38.561229 16258 master_runner.cc:315] Copying system tablet from 127.8.216.190:40249
I20260807 19:31:38.567471 16258 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.216.190:40249
I20260807 19:31:38.585206 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.568417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.8.216.190:40249)
0807 19:31:38.568690 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:38.568697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:38.569484 (+ 787us) server_negotiation.cc:407] Connection header received
0807 19:31:38.569678 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:38.569687 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:38.569783 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:38.569906 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:38.570981 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.572213 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.574367 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.575159 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.575332 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.577838 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:38.577845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:38.577851 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:38.577894 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:38.581698 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.582702 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.582708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.582712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.582887 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:38.583393 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:38.583399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:38.583403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:38.583785 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:38.583953 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:38.584564 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 19:31:38.584816 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 19:31:38.585944 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:38.568761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:38808)
0807 19:31:38.569344 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:38.569366 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:38.569528 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:38.569945 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:38.569950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:38.570491 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:38.570764 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.570776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.572389 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.572396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.574165 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:38.574182 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:38.575354 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:38.575361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:38.575576 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:38.576416 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:38.576441 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:38.577611 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:38.581886 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.581903 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.581910 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.582537 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.583030 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:38.583038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:38.583045 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:38.583262 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:38.583940 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:38.583948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:38.584228 (+ 280us) client_negotiation.cc:769] Sending connection context
0807 19:31:38.584434 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 19:31:38.584865 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":303,"threads_started":1}
I20260807 19:31:38.586665 15688 tablet_copy_service.cc:140] P 1eca52ce8d2d4ce4a2a2f52245054831: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cd37ceed4b604db5bbe24d6f070ae26d ({username='oryx', principal='oryx/127.8.216.188@KRBTEST.COM'} at 127.0.0.1:38808)
I20260807 19:31:38.586860 15688 tablet_copy_service.cc:161] P 1eca52ce8d2d4ce4a2a2f52245054831: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cd37ceed4b604db5bbe24d6f070ae26d at {username='oryx', principal='oryx/127.8.216.188@KRBTEST.COM'} at 127.0.0.1:38808: session id = cd37ceed4b604db5bbe24d6f070ae26d-00000000000000000000000000000000
I20260807 19:31:38.590744 15688 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:31:38.592921 16258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:31:38.600350 16258 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: tablet copy: Starting download of 0 data blocks...
I20260807 19:31:38.600730 16258 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: tablet copy: Starting download of 1 WAL segments...
I20260807 19:31:38.603502 16258 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:31:38.608039 15688 tablet_copy_service.cc:342] P 1eca52ce8d2d4ce4a2a2f52245054831: Request end of tablet copy session cd37ceed4b604db5bbe24d6f070ae26d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.216.188@KRBTEST.COM'} at 127.0.0.1:38808
I20260807 19:31:38.608224 15688 tablet_copy_service.cc:433] P 1eca52ce8d2d4ce4a2a2f52245054831: ending tablet copy session cd37ceed4b604db5bbe24d6f070ae26d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cd37ceed4b604db5bbe24d6f070ae26d
I20260807 19:31:38.621553 16258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:38.624980 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:38.625994 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:38.633358 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:38.635752 16258 server_base.cc:1034] running on GCE node
I20260807 19:31:38.636181 16258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:38.636897 16258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:38.638095 16258 hybrid_clock.cc:648] HybridClock initialized: now 1786131098638054 us; error 33 us; skew 500 ppm
I20260807 19:31:38.638352 16258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:38 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131098, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:38.640715 16258 init.cc:377] Logged in from keytab as oryx/127.8.216.188@KRBTEST.COM (short username oryx)
I20260807 19:31:38.648463 16258 webserver.cc:460] Webserver started at http://127.8.216.188:34339/ using document root <none> and password file <none>
I20260807 19:31:38.648869 16258 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:38.649075 16258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:38.652503 16258 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 19:31:38.654603 16479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:38.655094 16258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 19:31:38.655308 16258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
format_stamp: "Formatted at 2026-08-07 19:31:38 on dist-test-slave-8w9v"
server_key: "2c0342479437f4114077f60ba3830ba9"
server_key_iv: "d2fb8348db5830c1b635b44fbae0e10f"
server_key_version: "encryptionkey@0"
I20260807 19:31:38.655579 16258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:38.677352 16258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:38.682202 16258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:38.717098 16258 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.188:34395
I20260807 19:31:38.717200 16531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.188:34395 every 8 connection(s)
I20260807 19:31:38.718415 16258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
I20260807 19:31:38.725005 16532 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:31:38.729046 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Bootstrap starting.
I20260807 19:31:38.747535 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:31:38.748368 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: Bootstrap complete.
I20260807 19:31:38.749540 16532 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } }
I20260807 19:31:38.749969 16532 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cd37ceed4b604db5bbe24d6f070ae26d, State: Initialized, Role: LEARNER
I20260807 19:31:38.750260 16532 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } }
I20260807 19:31:38.751375 16535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:38.751663 16532 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:38.751801 16535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: LEARNER
I20260807 19:31:38.768234 16546 catalog_manager.cc:1259] Loaded cluster ID: ba522174f53942178942fa14e8246f0d
I20260807 19:31:38.768474 16546 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: loading cluster ID for follower catalog manager: success
I20260807 19:31:38.772411 16546 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: acquiring CA information for follower catalog manager: success
I20260807 19:31:38.775123 16546 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:31:39.013211 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.006082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.0.0.1:56828)
0807 19:31:39.006585 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.006602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.006714 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.007456 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.007461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.007491 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:39.007717 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.007727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.009110 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.009116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.010322 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.010335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.012256 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.012262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.012451 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.012530 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 19:31:39.012643 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:31:39.012814 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":252,"threads_started":1}
I20260807 19:31:39.013374 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.006379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.8.216.188:34395)
0807 19:31:39.007134 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.007138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.007158 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:31:39.007220 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.007223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.007294 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.007432 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:39.007886 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.008956 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.010491 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.012021 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.012208 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.012621 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:39.012682 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 19:31:39.012845 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":463,"threads_started":1}
I20260807 19:31:39.015156 16506 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260807 19:31:39.016244 16535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eca52ce8d2d4ce4a2a2f52245054831. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:39.016675 16535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: LEARNER
W20260807 19:31:39.026916 16464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:39.027285 16464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:39.027532 16464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:39.039338 16464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:39.039609 16464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:39.039897 16464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:39.054497 16464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:34139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=35659
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:39.057060 16464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:39.058858 16464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:39.079177 16554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:39.079391 16555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:39.079319 16557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:39.086081 16464 server_base.cc:1034] running on GCE node
I20260807 19:31:39.087050 16464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:39.088325 16464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:39.089572 16464 hybrid_clock.cc:648] HybridClock initialized: now 1786131099089513 us; error 49 us; skew 500 ppm
I20260807 19:31:39.089991 16464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:39.095786 16464 init.cc:377] Logged in from keytab as oryx/127.8.216.129@KRBTEST.COM (short username oryx)
I20260807 19:31:39.097682 16464 webserver.cc:460] Webserver started at http://127.8.216.129:35659/ using document root <none> and password file <none>
I20260807 19:31:39.098594 16464 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:39.098788 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:39.106426 16464 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:31:39.110410 16564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.112102 16464 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 19:31:39.112401 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "d7b2c05ea57242c193e09d98ec58c18a"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "6d1bb28df3f770ed5a894c6f798d0e49"
server_key_iv: "35b2a3a21f8b208190faa1996d4fa27b"
server_key_version: "encryptionkey@0"
I20260807 19:31:39.113741 16464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:39.144742 16464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:39.151178 16464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:39.151677 16464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:39.153465 16464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:39.155900 16464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:39.156193 16464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.156364 16464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:39.156508 16464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.248504 16464 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:34139
I20260807 19:31:39.248726 16683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:34139 every 8 connection(s)
I20260807 19:31:39.251662 16464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:39.261564 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16464
I20260807 19:31:39.262068 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15870
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.130@KRBTEST.COM
I20260807 19:31:39.297094 16688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.256594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49177 (local address 127.8.216.190:40249)
0807 19:31:39.257401 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.257406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.264459 (+ 7053us) server_negotiation.cc:407] Connection header received
0807 19:31:39.266981 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.266987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.267072 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.267143 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.270302 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.272075 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.276037 (+ 3962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.277474 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.277613 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.285872 (+ 8259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.285886 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.285890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.285930 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.290984 (+ 5054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.292366 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.292376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.292381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.292662 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.293483 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.293488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.293491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.294051 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.294203 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:39.296030 (+ 1827us) server_negotiation.cc:299] Negotiation successful
0807 19:31:39.296374 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":479,"threads_started":1}
I20260807 19:31:39.297232 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.261175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:33597 (local address 127.8.216.189:42159)
0807 19:31:39.272973 (+ 11798us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.272977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.272995 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:31:39.273049 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.273052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.273110 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.273252 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.275627 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.277177 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.280231 (+ 3054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.281332 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.281489 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.287900 (+ 6411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.287905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.287908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.287944 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.291600 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.292760 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.292765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.292769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.292952 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.293814 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.293823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.293828 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.294326 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.294501 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:39.296181 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0807 19:31:39.296470 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11656,"thread_start_us":11439,"threads_started":1}
I20260807 19:31:39.299593 16689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.260229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.8.216.129:33597)
0807 19:31:39.265416 (+ 5187us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.265441 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.266592 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.273280 (+ 6688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.273285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.274280 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.275432 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.275445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.277475 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.277481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.280020 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.280034 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.281921 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.281933 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.282254 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.283292 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.283316 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.287671 (+ 4355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.291765 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.291770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.291774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.292587 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.293107 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.293112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.293116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.293663 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.294593 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:39.294699 (+ 106us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:39.295352 (+ 653us) client_negotiation.cc:769] Sending connection context
0807 19:31:39.295809 (+ 457us) client_negotiation.cc:240] Negotiation successful
0807 19:31:39.298284 (+ 2475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4957,"mutex_wait_us":321,"spinlock_wait_cycles":42752,"thread_start_us":1287,"threads_started":1}
I20260807 19:31:39.299628 16690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.258464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:49177)
0807 19:31:39.262066 (+ 3602us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.262128 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.266592 (+ 4464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.267531 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.267540 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.268640 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.270000 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.270030 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.272397 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.272408 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.275786 (+ 3378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.275816 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.278157 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.278169 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.278636 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.279807 (+ 1171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.279858 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.285557 (+ 5699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.291221 (+ 5664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.291234 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.291258 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.292153 (+ 895us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.292836 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.292842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.292847 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.293322 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.294350 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:39.294361 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:39.295332 (+ 971us) client_negotiation.cc:769] Sending connection context
0807 19:31:39.297484 (+ 2152us) client_negotiation.cc:240] Negotiation successful
0807 19:31:39.298138 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2974,"mutex_wait_us":142,"spinlock_wait_cycles":448896,"thread_start_us":6099,"threads_started":1}
I20260807 19:31:39.301721 16685 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:39.301925 16686 heartbeater.cc:347] Connected to a master server at 127.8.216.189:42159
I20260807 19:31:39.302181 16685 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:39.302202 16686 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:39.303668 16685 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
I20260807 19:31:39.303802 16686 heartbeater.cc:511] Master 127.8.216.189:42159 requested a full tablet report, sending...
I20260807 19:31:39.305124 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.265886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:58761 (local address 127.8.216.188:34395)
0807 19:31:39.266095 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.266101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.270925 (+ 4824us) server_negotiation.cc:407] Connection header received
0807 19:31:39.272562 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.272570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.272689 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.272769 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.275424 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.276781 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.279814 (+ 3033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.280722 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.280875 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.292404 (+ 11529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.292413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.292430 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.292509 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.298013 (+ 5504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.301959 (+ 3946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.301966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.301969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.302272 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.303144 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.303152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.303156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.303585 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.303773 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:39.304416 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 19:31:39.304707 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:31:39.305898 15667 ts_manager.cc:194] Re-registered known tserver with Master: d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139)
I20260807 19:31:39.306331 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.265731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.8.216.129:58761)
0807 19:31:39.270785 (+ 5054us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.270805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.272340 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.273044 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.273050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.274280 (+ 1230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.275194 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.275212 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.276974 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.276979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.279488 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.279511 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.281473 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.281482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.281829 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.282839 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.282865 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.287138 (+ 4273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.300685 (+ 13547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.300690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.300694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.301366 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.302490 (+ 1124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.302495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.302498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.302975 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.303863 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:39.303868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:39.304033 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 19:31:39.305424 (+ 1391us) client_negotiation.cc:240] Negotiation successful
0807 19:31:39.305761 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4881,"mutex_wait_us":73,"thread_start_us":4641,"threads_started":1}
I20260807 19:31:39.307286 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:49177
I20260807 19:31:39.307700 16684 heartbeater.cc:347] Connected to a master server at 127.8.216.188:34395
I20260807 19:31:39.307925 16684 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:39.307950 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.264029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60000 (local address 127.8.216.130:39767)
0807 19:31:39.272026 (+ 7997us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.272037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.272070 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:31:39.273799 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.273814 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.274100 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.274272 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.278357 (+ 4085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.280939 (+ 2582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.283868 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.285077 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.285262 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.295124 (+ 9862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.295147 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.295169 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.295216 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.300906 (+ 5690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.303767 (+ 2861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.303777 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.303780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.304015 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.304591 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.304596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.304599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.305088 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.305705 (+ 617us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:39.306957 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0807 19:31:39.307298 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7783,"thread_start_us":7342,"threads_started":1}
I20260807 19:31:39.309100 16684 heartbeater.cc:511] Master 127.8.216.188:34395 requested a full tablet report, sending...
I20260807 19:31:39.309504 16298 ts_manager.cc:194] Registered new tserver with Master: d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139)
I20260807 19:31:39.310504 15998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:39.311408 16496 ts_manager.cc:194] Registered new tserver with Master: d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139)
I20260807 19:31:39.398301 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15870
I20260807 19:31:39.417635 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:39767
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:40249
--webserver_port=42485
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:39.482539 16547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: NON_VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: true } } }
I20260807 19:31:39.482946 16547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:39.484587 16697 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: attempting to promote NON_VOTER cd37ceed4b604db5bbe24d6f070ae26d to VOTER
I20260807 19:31:39.485388 16697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } }
I20260807 19:31:39.488066 16506 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 LEARNER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:31:39.488437 16308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Refusing update from remote peer 1eca52ce8d2d4ce4a2a2f52245054831: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:31:39.488842 16547 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:31:39.489434 16697 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:31:39.492995 16547 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.494777 16308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.496466 16547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.496697 16547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:39.497272 16547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.497534 16547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:39.497812 16547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.498041 16547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:39.497566 16506 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.500044 16703 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.500560 16703 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:39.502823 16535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:39.503139 16535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: FOLLOWER
W20260807 19:31:39.846653 16696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:39.846936 16696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:39.847033 16696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:39.854662 16696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:39.854815 16696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:39.855001 16696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:39.865345 16696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:39767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=42485
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:39.866755 16696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:39.868100 16696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:39.882069 16709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:39.882136 16708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:39.882104 16712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:39.883361 16696 server_base.cc:1034] running on GCE node
I20260807 19:31:39.884038 16696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:39.885022 16696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:39.886229 16696 hybrid_clock.cc:648] HybridClock initialized: now 1786131099886178 us; error 46 us; skew 500 ppm
I20260807 19:31:39.886561 16696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:39.889899 16696 init.cc:377] Logged in from keytab as oryx/127.8.216.130@KRBTEST.COM (short username oryx)
I20260807 19:31:39.891345 16696 webserver.cc:460] Webserver started at http://127.8.216.130:42485/ using document root <none> and password file <none>
I20260807 19:31:39.892024 16696 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:39.892231 16696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:39.897692 16696 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:39.900794 16720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.902168 16696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:31:39.902303 16696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "1f0ecec2bb45417d9f28252354030dc6"
format_stamp: "Formatted at 2026-08-07 19:31:35 on dist-test-slave-8w9v"
server_key: "2d4391561631b4d2ae6c267afdb1f06f"
server_key_iv: "0993aafe9363c81c1550f115e4a7e188"
server_key_version: "encryptionkey@0"
I20260807 19:31:39.903388 16696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:39.923604 16696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:39.928328 16696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:39.928580 16696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:39.929553 16696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:39.930910 16696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:39.931008 16696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.931087 16696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:39.931143 16696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:39.966636 16696 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:39767
I20260807 19:31:39.966697 16836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:39767 every 8 connection(s)
I20260807 19:31:39.968312 16696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:39.975572 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16696
I20260807 19:31:39.975884 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16010
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
Aug 07 19:31:39 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131099, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
I20260807 19:31:40.000958 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.971432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:52545 (local address 127.8.216.190:40249)
0807 19:31:39.972076 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.972081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.974825 (+ 2744us) server_negotiation.cc:407] Connection header received
0807 19:31:39.977130 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.977139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.977261 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.977392 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.980704 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.981930 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.985045 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.985706 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.985811 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.992417 (+ 6606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.992424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.992428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.992487 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.996932 (+ 4445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.997810 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.997816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.997820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.997975 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998768 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.998774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.998777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.999208 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.999340 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.000320 (+ 980us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.000569 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":365,"threads_started":1}
I20260807 19:31:40.001029 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.972510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:57791 (local address 127.8.216.189:42159)
0807 19:31:39.973058 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.973067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.973877 (+ 810us) server_negotiation.cc:407] Connection header received
0807 19:31:39.977206 (+ 3329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.977215 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.977346 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.977447 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.980704 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.981922 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.985044 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.985707 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.985811 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.992405 (+ 6594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.992413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.992417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.992476 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.996684 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.997800 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.997806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.997809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.997985 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998623 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.998628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.998631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.999017 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:39.999162 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.000281 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.000550 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":261,"threads_started":1}
I20260807 19:31:40.002688 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.972145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.8.216.130:57791)
0807 19:31:39.973697 (+ 1552us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.973753 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.977028 (+ 3275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.977968 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.977985 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.979308 (+ 1323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.980470 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.980499 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.982162 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.982168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.984857 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.984868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.986254 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.986263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.986797 (+ 534us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.987837 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.987883 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.992050 (+ 4167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.996871 (+ 4821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.996880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.996898 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.997641 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.998133 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.998142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.998466 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.999271 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:39.999282 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:39.999817 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.001238 (+ 1421us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.001541 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1228,"mutex_wait_us":587,"spinlock_wait_cycles":38528,"thread_start_us":990,"threads_started":1}
I20260807 19:31:40.002713 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.974742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.8.216.130:53783)
0807 19:31:39.975341 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.975366 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.976637 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.980658 (+ 4021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.980666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.981388 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.981811 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.981827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.984232 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.984240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.986754 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.986776 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.988297 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.988307 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.988624 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.989375 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.989409 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.993178 (+ 3769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.998210 (+ 5032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.998219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.998778 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.999276 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.999281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.999284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.999548 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.000255 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:40.000260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:40.000420 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.000571 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.001175 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":93,"thread_start_us":320,"threads_started":1}
I20260807 19:31:40.003504 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.974878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:53783 (local address 127.8.216.188:34395)
0807 19:31:39.975836 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.975844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.975866 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:31:39.980302 (+ 4436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.980312 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.980439 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.982034 (+ 1595us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.982443 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.983967 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.987020 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.987990 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.989093 (+ 1103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.993640 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:39.993649 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:39.993657 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:39.993724 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:39.998051 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998959 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.998964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.998968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:39.999129 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:39.999728 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:39.999733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:39.999736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.000082 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.002689 (+ 2607us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.002848 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.003095 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":1271,"threads_started":1}
I20260807 19:31:40.004030 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.972145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:52545)
0807 19:31:39.974628 (+ 2483us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:39.974647 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:39.976647 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:39.977968 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:39.977986 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:39.979373 (+ 1387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:39.980470 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.980499 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.982200 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.982204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.984852 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:39.984868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:39.986254 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:39.986263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:39.986796 (+ 533us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:39.987904 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:39.987924 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:39.992050 (+ 4126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:39.997097 (+ 5047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.997102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.997106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.997672 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.998133 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:39.998138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:39.998142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:39.998626 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:39.999408 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:39.999416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:39.999766 (+ 350us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.001688 (+ 1922us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.001986 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2305,"mutex_wait_us":566,"spinlock_wait_cycles":45312,"thread_start_us":1901,"threads_started":1}
I20260807 19:31:40.004971 16837 heartbeater.cc:347] Connected to a master server at 127.8.216.188:34395
I20260807 19:31:40.005048 16839 heartbeater.cc:347] Connected to a master server at 127.8.216.189:42159
I20260807 19:31:40.005321 16839 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.005326 16837 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.005766 16838 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:40.005906 16838 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.006575 16837 heartbeater.cc:511] Master 127.8.216.188:34395 requested a full tablet report, sending...
I20260807 19:31:40.006597 16839 heartbeater.cc:511] Master 127.8.216.189:42159 requested a full tablet report, sending...
I20260807 19:31:40.008701 16496 ts_manager.cc:194] Registered new tserver with Master: 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:40.008699 16298 ts_manager.cc:194] Registered new tserver with Master: 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:40.009946 16838 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
Aug 07 19:31:40 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.131@KRBTEST.COM
I20260807 19:31:40.011018 15667 ts_manager.cc:194] Re-registered known tserver with Master: 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:40.011873 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:52545
I20260807 19:31:40.019098 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:39.976963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.8.216.131:39819)
0807 19:31:39.995167 (+ 18204us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:39.995177 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:39.995210 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:31:39.995283 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:39.995292 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:39.995590 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:39.998801 (+ 3211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:39.999088 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.000685 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.002822 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.003680 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.004508 (+ 828us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.012557 (+ 8049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.012578 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.012598 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.012658 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.016028 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.016823 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.016830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.016832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.016983 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.017367 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.017370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.017372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.017697 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.017938 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.018411 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.018711 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18005,"thread_start_us":17830,"threads_started":1}
I20260807 19:31:40.020936 16142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:40.106326 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16010
I20260807 19:31:40.124908 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:39819
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:40249
--webserver_port=42415
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:40.311863 16685 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
W20260807 19:31:40.467868 16852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:40.468246 16852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:40.468360 16852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:40.475827 16852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:40.475960 16852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:40.476101 16852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:40.486213 16852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:39819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=42415
--tserver_master_addrs=127.8.216.190:40249,127.8.216.189:42159,127.8.216.188:34395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:40.487511 16852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:40.488891 16852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:40.502548 16857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:40.502596 16859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:40.502511 16861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:40.504033 16852 server_base.cc:1034] running on GCE node
I20260807 19:31:40.504823 16852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:40.505913 16852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:40.507110 16852 hybrid_clock.cc:648] HybridClock initialized: now 1786131100507047 us; error 60 us; skew 500 ppm
I20260807 19:31:40.507534 16852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:40 dist-test-slave-8w9v krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131100, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:40.510782 16852 init.cc:377] Logged in from keytab as oryx/127.8.216.131@KRBTEST.COM (short username oryx)
I20260807 19:31:40.512421 16852 webserver.cc:460] Webserver started at http://127.8.216.131:42415/ using document root <none> and password file <none>
I20260807 19:31:40.513134 16852 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:40.513293 16852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:40.519455 16852 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 19:31:40.522286 16868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:40.523705 16852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:40.523880 16852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "6a6a28a6844f4794a7831fba938ae07e"
format_stamp: "Formatted at 2026-08-07 19:31:36 on dist-test-slave-8w9v"
server_key: "fb24cfa5c959facc9f0936ab881ee355"
server_key_iv: "cf96956428e8dda7e919af366125be1e"
server_key_version: "encryptionkey@0"
I20260807 19:31:40.524988 16852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:40.558483 16852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:40.563196 16852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:40.563527 16852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:40.564638 16852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:40.566077 16852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:40.566174 16852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:40.566332 16852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:40.566445 16852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:40.600943 16852 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:39819
I20260807 19:31:40.600941 16985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:39819 every 8 connection(s)
I20260807 19:31:40.602516 16852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:40.610834 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 16852
Aug 07 19:31:40 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131100, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.188@KRBTEST.COM
Aug 07 19:31:40 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131100, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:40.664597 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.617512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:57453 (local address 127.8.216.188:34395)
0807 19:31:40.637360 (+ 19848us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.637368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.637392 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:31:40.637473 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.637478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.637581 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.637700 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.640916 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.642548 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.646056 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.646980 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.647114 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.655011 (+ 7897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.655022 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.655030 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.655089 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.659358 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.660919 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.660928 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.660934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.661137 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.662233 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.662239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.662242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.662626 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.662800 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.663834 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.664148 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19702,"thread_start_us":19585,"threads_started":1}
Aug 07 19:31:40 dist-test-slave-8w9v krb5kdc[15619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131100, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.189@KRBTEST.COM
I20260807 19:31:40.668783 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.620297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.8.216.131:57453)
0807 19:31:40.636670 (+ 16373us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.636697 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.636854 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.638273 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.638288 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.639431 (+ 1143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:40.640668 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.640700 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.642837 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.642845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.645802 (+ 2957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.645826 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.647528 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.647536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.647834 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.648760 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:40.648802 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:40.653819 (+ 5017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:40.659555 (+ 5736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.659567 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.659593 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.660323 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.661339 (+ 1016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.661345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.661354 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.661621 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.662880 (+ 1259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:40.662890 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:40.663356 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.666426 (+ 3070us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.667051 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16212,"mutex_wait_us":67,"thread_start_us":16038,"threads_started":1}
I20260807 19:31:40.680953 16986 heartbeater.cc:347] Connected to a master server at 127.8.216.188:34395
I20260807 19:31:40.681375 16986 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.682358 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.606376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:47771)
0807 19:31:40.614532 (+ 8156us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.614624 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.629949 (+ 15325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.638273 (+ 8324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.638288 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.639684 (+ 1396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:40.640742 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.640761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.650007 (+ 9246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.650015 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.651690 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.651710 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.658429 (+ 6719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.658436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.658645 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.659228 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:40.659253 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:40.662320 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:40.679032 (+ 16712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.679039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.679043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.679616 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.680223 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.680229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.680232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.680463 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.681171 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:40.681175 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:40.681363 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.681524 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.681849 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7690,"mutex_wait_us":34,"spinlock_wait_cycles":1408,"thread_start_us":9392,"threads_started":1}
I20260807 19:31:40.682435 16986 heartbeater.cc:511] Master 127.8.216.188:34395 requested a full tablet report, sending...
I20260807 19:31:40.683403 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.606376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.8.216.131:60989)
0807 19:31:40.634620 (+ 28244us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.634642 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.634766 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.639440 (+ 4674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.639444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.640112 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:40.640668 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.640700 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.651157 (+ 10457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.651162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.653334 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.653348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.656534 (+ 3186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.656543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.656749 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.658883 (+ 2134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:40.659331 (+ 448us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:40.670657 (+ 11326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:40.680300 (+ 9643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.680307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.680313 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.680897 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.681429 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:40.681435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:40.681438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:40.681630 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:40.682334 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:40.682340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:40.682541 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.682658 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.682958 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28058,"mutex_wait_us":70,"spinlock_wait_cycles":75648,"thread_start_us":27737,"threads_started":1}
I20260807 19:31:40.684583 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.605883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:60989 (local address 127.8.216.189:42159)
0807 19:31:40.638954 (+ 33071us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.638961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.638984 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:31:40.639048 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.639052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.639133 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.639241 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.647249 (+ 8008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.650931 (+ 3682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.653756 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.654657 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.654820 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.670999 (+ 16179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.671009 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.671017 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.671078 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.680071 (+ 8993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.681085 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.681092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.681097 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.681272 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.681792 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.681798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.681803 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.682180 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.683677 (+ 1497us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.683830 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.684107 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32928,"thread_start_us":32806,"threads_started":1}
I20260807 19:31:40.684914 16496 ts_manager.cc:194] Registered new tserver with Master: 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819)
I20260807 19:31:40.685631 16989 heartbeater.cc:347] Connected to a master server at 127.8.216.189:42159
I20260807 19:31:40.685775 16989 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.686132 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.605599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:47771 (local address 127.8.216.190:40249)
0807 19:31:40.637639 (+ 32040us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.637647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.637670 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:31:40.637733 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.637737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.637809 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.637920 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.640914 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.649820 (+ 8906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.655177 (+ 5357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.658204 (+ 3027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.658344 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.662623 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.662634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.662642 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.662699 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.678810 (+ 16111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.679812 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.679818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.679824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.680037 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.680621 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.680625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.680628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.681021 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.683634 (+ 2613us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.685449 (+ 1815us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.685733 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31871,"thread_start_us":31869,"threads_started":1}
I20260807 19:31:40.686277 16989 heartbeater.cc:511] Master 127.8.216.189:42159 requested a full tablet report, sending...
I20260807 19:31:40.687567 16298 ts_manager.cc:194] Registered new tserver with Master: 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819)
I20260807 19:31:40.687896 16987 heartbeater.cc:347] Connected to a master server at 127.8.216.190:40249
I20260807 19:31:40.688043 16987 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:40.688496 16987 heartbeater.cc:511] Master 127.8.216.190:40249 requested a full tablet report, sending...
I20260807 19:31:40.689882 15667 ts_manager.cc:194] Re-registered known tserver with Master: 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819)
I20260807 19:31:40.690150 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.640938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.8.216.190:40249)
0807 19:31:40.655706 (+ 14768us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.655713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.656740 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 19:31:40.657161 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.657170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.657281 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.657401 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.661426 (+ 4025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.662469 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.664340 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.665007 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.665124 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.671877 (+ 6753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.671884 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.671888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.671935 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.684814 (+ 12879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.686792 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.686799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.686803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.686989 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.687523 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.687529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.687534 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.687906 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.689130 (+ 1224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.689275 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.689564 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14354,"thread_start_us":14535,"threads_started":1}
I20260807 19:31:40.691147 15667 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:47771
I20260807 19:31:40.691701 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.651265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.8.216.188:34395)
0807 19:31:40.665221 (+ 13956us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.665227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.665980 (+ 753us) server_negotiation.cc:407] Connection header received
0807 19:31:40.666038 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.666044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.666122 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.666257 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.670731 (+ 4474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.671909 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.674520 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.675224 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.675323 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.683058 (+ 7735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.683066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.683071 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.683118 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.687963 (+ 4845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.688789 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.688797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.688803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.688965 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.689468 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.689477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.689480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.689861 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.690156 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.691087 (+ 931us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.691332 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13833,"thread_start_us":24876,"threads_started":1}
I20260807 19:31:40.694908 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.641274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.8.216.189:42159)
0807 19:31:40.666647 (+ 25373us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.666657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.666685 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:40.666760 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.666768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.666864 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.667023 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:40.672660 (+ 5637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.674211 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.676385 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.677872 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.678028 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.682685 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:40.682692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:40.682697 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:40.682734 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:40.690452 (+ 7718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.692466 (+ 2014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.692471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.692474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.692633 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:40.693109 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:40.693116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:40.693120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:40.693534 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:40.693805 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.694203 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.694460 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25183,"thread_start_us":25979,"threads_started":1}
I20260807 19:31:40.695456 15666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:40.702798 15666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:40.760335 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.732248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:39767 (local address 127.0.0.1:60010)
0807 19:31:40.752686 (+ 20438us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.752788 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.752937 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.753699 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.753704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.753732 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:40.754124 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.754142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.755684 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.755695 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.758063 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.758072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.759465 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.759472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.759641 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.759681 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.759800 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.759936 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20322,"thread_start_us":20173,"threads_started":1}
I20260807 19:31:40.760383 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.732827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.8.216.130:39767)
0807 19:31:40.749096 (+ 16269us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.749110 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.752910 (+ 3800us) server_negotiation.cc:407] Connection header received
0807 19:31:40.753184 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.753196 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.753480 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.753720 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:40.754366 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.755501 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.758260 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.759219 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.759392 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.759628 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.759875 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.760064 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15984,"thread_start_us":16261,"threads_started":1}
I20260807 19:31:40.763599 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.731648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39819 (local address 127.0.0.1:40106)
0807 19:31:40.750192 (+ 18544us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.750219 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.750397 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.752091 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.752099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.752148 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:40.752721 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.752733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.755508 (+ 2775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.755513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.758037 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.758054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.762733 (+ 4679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.762756 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.762929 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.762964 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.763059 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.763199 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18306,"thread_start_us":17937,"threads_started":1}
I20260807 19:31:40.763819 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.731918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.8.216.131:39819)
0807 19:31:40.751383 (+ 19465us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.751390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.751419 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:31:40.751505 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.751511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.751812 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.752083 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:40.752955 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.755372 (+ 2417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.759582 (+ 4210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.762374 (+ 2792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.762623 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.762909 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.763160 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.763370 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19291,"thread_start_us":19111,"threads_started":1}
I20260807 19:31:40.765822 16770 tablet_service.cc:1467] Processing CreateTablet for tablet bd175cbe961a4e878be48db0c55a7628 (DEFAULT_TABLE table=test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:40.767670 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd175cbe961a4e878be48db0c55a7628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:40.768167 16919 tablet_service.cc:1467] Processing CreateTablet for tablet bd175cbe961a4e878be48db0c55a7628 (DEFAULT_TABLE table=test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:40.769645 16919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd175cbe961a4e878be48db0c55a7628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:40.771776 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.730707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40798)
0807 19:31:40.731370 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.731392 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.731517 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.754112 (+ 22595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.754117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.754144 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:40.754571 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.754582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.756622 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.756629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.759256 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.759274 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.770064 (+ 10790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.770083 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.770494 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.770554 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.770658 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.770887 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":352,"threads_started":1}
I20260807 19:31:40.773717 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.731019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.8.216.129:34139)
0807 19:31:40.753553 (+ 22534us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.753562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.753590 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:40.753692 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.753699 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.753885 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.754784 (+ 899us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:40.755034 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.756474 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.767532 (+ 11058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.769565 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.772326 (+ 2761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.772669 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.772812 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.773099 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22347,"thread_start_us":22144,"threads_started":1}
I20260807 19:31:40.779358 16615 tablet_service.cc:1467] Processing CreateTablet for tablet bd175cbe961a4e878be48db0c55a7628 (DEFAULT_TABLE table=test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:40.781165 16615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd175cbe961a4e878be48db0c55a7628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:40.790194 17024 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Bootstrap starting.
I20260807 19:31:40.792297 17025 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Bootstrap starting.
I20260807 19:31:40.795776 17024 tablet_bootstrap.cc:654] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:40.796800 17025 tablet_bootstrap.cc:654] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:40.798061 17024 log.cc:824] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:40.798522 17025 log.cc:824] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:40.802693 17024 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: No bootstrap required, opened a new log
I20260807 19:31:40.802733 17028 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Bootstrap starting.
I20260807 19:31:40.803067 17024 ts_tablet_manager.cc:1397] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260807 19:31:40.803089 17025 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: No bootstrap required, opened a new log
I20260807 19:31:40.803536 17025 ts_tablet_manager.cc:1397] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260807 19:31:40.809453 17028 tablet_bootstrap.cc:654] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:40.811851 17028 log.cc:824] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:40.815662 17025 raft_consensus.cc:348] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.816227 17025 raft_consensus.cc:374] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:40.816395 17025 raft_consensus.cc:729] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6a28a6844f4794a7831fba938ae07e, State: Initialized, Role: FOLLOWER
I20260807 19:31:40.816730 17028 tablet_bootstrap.cc:492] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: No bootstrap required, opened a new log
I20260807 19:31:40.817160 17028 ts_tablet_manager.cc:1397] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260807 19:31:40.817221 17025 consensus_queue.cc:260] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.818637 17024 raft_consensus.cc:348] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.819300 17024 raft_consensus.cc:374] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:40.819547 17024 raft_consensus.cc:729] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0ecec2bb45417d9f28252354030dc6, State: Initialized, Role: FOLLOWER
I20260807 19:31:40.820048 17025 ts_tablet_manager.cc:1428] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260807 19:31:40.820389 16987 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
I20260807 19:31:40.820492 17024 consensus_queue.cc:260] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.828302 16838 heartbeater.cc:503] Master 127.8.216.190:40249 was elected leader, sending a full tablet report...
I20260807 19:31:40.829406 17024 ts_tablet_manager.cc:1428] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Time spent starting tablet: real 0.026s user 0.021s sys 0.005s
I20260807 19:31:40.836812 17028 raft_consensus.cc:348] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.837282 17028 raft_consensus.cc:374] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:40.837427 17028 raft_consensus.cc:729] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b2c05ea57242c193e09d98ec58c18a, State: Initialized, Role: FOLLOWER
I20260807 19:31:40.838068 17028 consensus_queue.cc:260] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.840615 17028 ts_tablet_manager.cc:1428] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Time spent starting tablet: real 0.023s user 0.015s sys 0.005s
W20260807 19:31:40.855907 16990 tablet.cc:2369] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:40.908373 17032 raft_consensus.cc:482] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:40.908779 17032 raft_consensus.cc:504] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.911499 17032 leader_election.cc:288] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819), 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:40.921892 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.912438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:46363 (local address 127.8.216.130:39767)
0807 19:31:40.912795 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.912805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.914569 (+ 1764us) server_negotiation.cc:407] Connection header received
0807 19:31:40.914889 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.914895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.914973 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.915074 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:40.915976 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.916865 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.919675 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.920791 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.921002 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.921088 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.921426 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.921560 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:31:40.922087 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.911635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:39767 (local address 127.8.216.129:46363)
0807 19:31:40.914440 (+ 2805us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.914459 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.914734 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.915098 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.915103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.915127 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:40.915760 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.915772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.917049 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.917053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.919499 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.919512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.921057 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.921063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.921247 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.921272 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.921386 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.921528 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2616,"mutex_wait_us":61,"thread_start_us":2226,"threads_started":1}
I20260807 19:31:40.922366 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.911639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:52539 (local address 127.8.216.131:39819)
0807 19:31:40.911913 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.911923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.914750 (+ 2827us) server_negotiation.cc:407] Connection header received
0807 19:31:40.914935 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.914947 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.915009 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.915111 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:40.916018 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.916946 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.919894 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.920955 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.921170 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.921277 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.921685 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.921812 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:31:40.922608 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.912415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39819 (local address 127.8.216.129:52539)
0807 19:31:40.914615 (+ 2200us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:40.914647 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:40.914773 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:40.915129 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:40.915133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:40.915157 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:40.915771 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.915780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.917118 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.917122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.919730 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:40.919743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.921325 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.921332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:40.921516 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.921540 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:31:40.921645 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:31:40.921778 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2020,"mutex_wait_us":24,"thread_start_us":1662,"threads_started":1}
I20260807 19:31:40.923935 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd175cbe961a4e878be48db0c55a7628" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6a28a6844f4794a7831fba938ae07e" is_pre_election: true
I20260807 19:31:40.923956 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd175cbe961a4e878be48db0c55a7628" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0ecec2bb45417d9f28252354030dc6" is_pre_election: true
I20260807 19:31:40.924811 16940 raft_consensus.cc:2454] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 0.
I20260807 19:31:40.924861 16792 raft_consensus.cc:2454] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 0.
I20260807 19:31:40.925742 16565 leader_election.cc:302] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f0ecec2bb45417d9f28252354030dc6, 6a6a28a6844f4794a7831fba938ae07e, d7b2c05ea57242c193e09d98ec58c18a; no voters:
I20260807 19:31:40.926249 17032 raft_consensus.cc:2781] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:40.926414 17032 raft_consensus.cc:482] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:40.926558 17032 raft_consensus.cc:3037] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:40.929767 17032 raft_consensus.cc:504] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.930555 17032 leader_election.cc:288] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819), 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:40.931113 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd175cbe961a4e878be48db0c55a7628" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6a28a6844f4794a7831fba938ae07e"
I20260807 19:31:40.931236 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd175cbe961a4e878be48db0c55a7628" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0ecec2bb45417d9f28252354030dc6"
I20260807 19:31:40.931368 16940 raft_consensus.cc:3037] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:40.931597 16792 raft_consensus.cc:3037] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:40.935074 16940 raft_consensus.cc:2454] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 1.
I20260807 19:31:40.935555 16565 leader_election.cc:302] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6a28a6844f4794a7831fba938ae07e, d7b2c05ea57242c193e09d98ec58c18a; no voters:
I20260807 19:31:40.935611 16792 raft_consensus.cc:2454] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 1.
I20260807 19:31:40.935994 17032 raft_consensus.cc:2781] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:40.936722 17032 raft_consensus.cc:686] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 1 LEADER]: Becoming Leader. State: Replica: d7b2c05ea57242c193e09d98ec58c18a, State: Running, Role: LEADER
I20260807 19:31:40.937407 17032 consensus_queue.cc:237] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:40.943513 15666 catalog_manager.cc:5495] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a reported cstate change: term changed from 0 to 1, leader changed from <none> to d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d7b2c05ea57242c193e09d98ec58c18a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } health_report { overall_health: UNKNOWN } } }
W20260807 19:31:40.972368 16840 tablet.cc:2369] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:40.994338 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:40.988248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.8.216.129:34139)
0807 19:31:40.988462 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:40.988467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:40.988486 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:31:40.988596 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:40.988600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:40.988693 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:40.988820 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:40.989269 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.990137 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.991873 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:40.992565 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:40.992660 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:40.993190 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:40.993705 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:40.993817 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:40.993912 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:31:40.993992 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:31:41.005455 16792 raft_consensus.cc:1261] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Refusing update from remote peer d7b2c05ea57242c193e09d98ec58c18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:41.005589 16940 raft_consensus.cc:1261] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Refusing update from remote peer d7b2c05ea57242c193e09d98ec58c18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:41.006726 17035 consensus_queue.cc:1035] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:41.007304 17032 consensus_queue.cc:1035] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:31:41.018179 16687 tablet.cc:2369] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:41.055769 15666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:41.056366 15666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:41.065884 15666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1eca52ce8d2d4ce4a2a2f52245054831: Sending DeleteTablet for 3 replicas of tablet bd175cbe961a4e878be48db0c55a7628
I20260807 19:31:41.067730 16919 tablet_service.cc:1514] Processing DeleteTablet for tablet bd175cbe961a4e878be48db0c55a7628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:41 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40106
I20260807 19:31:41.067881 16615 tablet_service.cc:1514] Processing DeleteTablet for tablet bd175cbe961a4e878be48db0c55a7628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:41 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40798
I20260807 19:31:41.068681 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet bd175cbe961a4e878be48db0c55a7628 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:41 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:60010
I20260807 19:31:41.070905 17053 tablet_replica.cc:330] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: stopping tablet replica
I20260807 19:31:41.071610 17053 raft_consensus.cc:2229] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:41.072563 17053 raft_consensus.cc:2258] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:41.075317 17053 ts_tablet_manager.cc:1905] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:41.083307 17050 tablet_replica.cc:330] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: stopping tablet replica
I20260807 19:31:41.083916 17050 raft_consensus.cc:2229] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:41.087527 17053 ts_tablet_manager.cc:1918] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:41.087555 17050 raft_consensus.cc:2258] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:41.088238 17053 log.cc:1196] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/bd175cbe961a4e878be48db0c55a7628
I20260807 19:31:41.089744 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 15633
I20260807 19:31:41.090632 17053 ts_tablet_manager.cc:1939] T bd175cbe961a4e878be48db0c55a7628 P 6a6a28a6844f4794a7831fba938ae07e: Deleting consensus metadata
I20260807 19:31:41.091886 15698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:41.091928 17050 ts_tablet_manager.cc:1905] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:41.092343 17052 tablet_replica.cc:330] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: stopping tablet replica
I20260807 19:31:41.093581 17052 raft_consensus.cc:2229] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:41.094447 17052 raft_consensus.cc:2258] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:41.097465 17052 ts_tablet_manager.cc:1905] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:41.104074 17050 ts_tablet_manager.cc:1918] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:41.104446 17050 log.cc:1196] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/bd175cbe961a4e878be48db0c55a7628
I20260807 19:31:41.105360 17050 ts_tablet_manager.cc:1939] T bd175cbe961a4e878be48db0c55a7628 P d7b2c05ea57242c193e09d98ec58c18a: Deleting consensus metadata
I20260807 19:31:41.109045 17052 ts_tablet_manager.cc:1918] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:41.109301 17052 log.cc:1196] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/bd175cbe961a4e878be48db0c55a7628
I20260807 19:31:41.109912 17052 ts_tablet_manager.cc:1939] T bd175cbe961a4e878be48db0c55a7628 P 1f0ecec2bb45417d9f28252354030dc6: Deleting consensus metadata
I20260807 19:31:41.127631 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:41.070444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.8.216.189:42159)
0807 19:31:41.070732 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:41.070739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:41.087153 (+ 16414us) server_negotiation.cc:407] Connection header received
0807 19:31:41.090398 (+ 3245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:41.090405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:41.090531 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:41.090642 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:41.098453 (+ 7811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.100194 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.104048 (+ 3854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.105089 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.105228 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:41.117105 (+ 11877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:41.117114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:41.117119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:41.117176 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:41.123088 (+ 5912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.124380 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.124387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.124391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.124600 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.125237 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.125242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.125245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.125822 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:41.126211 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:41.126689 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 19:31:41.127038 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:31:41.152268 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:41.087881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.8.216.188:34395)
0807 19:31:41.088102 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:41.088108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:41.127493 (+ 39385us) server_negotiation.cc:407] Connection header received
0807 19:31:41.128418 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:41.128423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:41.128521 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:41.128615 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:41.131870 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.133645 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.136255 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.137198 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.137343 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:41.142039 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:41.142049 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:41.142054 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:41.142145 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:41.147965 (+ 5820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.149178 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.149184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.149188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.149499 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.150123 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.150129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.150132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.150694 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:41.151082 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:41.151523 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 19:31:41.151808 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:31:41.261183 15653 catalog_manager.cc:4841] TS 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767): tablet bd175cbe961a4e878be48db0c55a7628 (table test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]) successfully deleted
I20260807 19:31:41.262266 15654 catalog_manager.cc:4841] TS d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139): tablet bd175cbe961a4e878be48db0c55a7628 (table test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]) successfully deleted
I20260807 19:31:41.263309 15651 catalog_manager.cc:4841] TS 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819): tablet bd175cbe961a4e878be48db0c55a7628 (table test-table [id=25eb5ccd3e744258a7c5bd9e34d8e801]) successfully deleted
W20260807 19:31:41.263491 15651 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.216.131:39819, user_credentials={real_user=slave}} blocked reactor thread for 170452us
I20260807 19:31:41.264225 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 15633
I20260807 19:31:41.327782 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:41.307574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.8.216.189:42159)
0807 19:31:41.307914 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:41.307924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:41.309248 (+ 1324us) server_negotiation.cc:407] Connection header received
0807 19:31:41.309553 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:41.309560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:41.309651 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:41.309728 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:41.313006 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.314221 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.316325 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.317044 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.317147 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:41.321439 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:41.321446 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:41.321450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:41.321501 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:41.324885 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.325724 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.325730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.325734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.325871 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.326286 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.326295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.326297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.326581 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:41.326885 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:41.327275 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:31:41.327491 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:31:41.331389 16996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:41.308984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.8.216.188:34395)
0807 19:31:41.309237 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:41.309245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:41.312882 (+ 3637us) server_negotiation.cc:407] Connection header received
0807 19:31:41.313659 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:41.313665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:41.313762 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:41.313867 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:41.315649 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.316548 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.317921 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:41.318721 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:41.318868 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:41.321648 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:41.321655 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:41.321659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:41.321703 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:41.327009 (+ 5306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.328912 (+ 1903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.328918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.328921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.329531 (+ 610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:41.329919 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:41.329923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:41.329925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:41.330212 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:41.330416 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:41.330791 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 19:31:41.330974 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:31:42.036569 17064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.035263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:33545)
0807 19:31:42.036099 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.036307 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":536,"threads_started":1}
W20260807 19:31:42.036998 16987 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:40249 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.042614 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.041667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:56101)
0807 19:31:42.042253 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.042413 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":258,"threads_started":1}
W20260807 19:31:42.043242 16838 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:40249 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.048895 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.047895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:36263)
0807 19:31:42.048484 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.048655 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":330,"threads_started":1}
W20260807 19:31:42.049353 16685 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:40249 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.566053 17067 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1eca52ce8d2d4ce4a2a2f52245054831)
I20260807 19:31:42.566390 17067 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } }
I20260807 19:31:42.566866 17068 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1eca52ce8d2d4ce4a2a2f52245054831)
I20260807 19:31:42.567059 17068 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } }
I20260807 19:31:42.568018 17067 leader_election.cc:288] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249), cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188:34395)
I20260807 19:31:42.569613 17068 leader_election.cc:288] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249), bd2f295b92284022a3164d79b48f824a (127.8.216.189:42159)
I20260807 19:31:42.570423 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.568875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39262)
0807 19:31:42.569974 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.570165 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":822,"threads_started":1}
I20260807 19:31:42.571017 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.568405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39230)
0807 19:31:42.570248 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.570564 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1697,"thread_start_us":1792,"threads_started":1}
W20260807 19:31:42.571403 16482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
W20260807 19:31:42.573019 16282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.573652 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.573224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39266)
0807 19:31:42.573392 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.573497 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 19:31:42.573839 16482 leader_election.cc:334] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249): Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.577657 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.570081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:34395 (local address 127.0.0.1:56876)
0807 19:31:42.570632 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.570658 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:42.570767 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:42.571655 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:42.571663 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:42.571693 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:42.571951 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.571970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.573679 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.573686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.575152 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.575169 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.576728 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.576741 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.576991 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.577030 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:31:42.577148 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:31:42.577328 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":2037,"threads_started":1}
I20260807 19:31:42.577747 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.577221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39270)
0807 19:31:42.577391 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.577555 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 19:31:42.577996 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.570370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.8.216.188:34395)
0807 19:31:42.571261 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.571267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.571283 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:42.571338 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.571342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.571450 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.571608 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:42.572192 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.573508 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.575332 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.576536 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.576686 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.576801 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.577197 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.577341 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":1040,"threads_started":1}
W20260807 19:31:42.577994 16282 leader_election.cc:334] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249): Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.578685 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd2f295b92284022a3164d79b48f824a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" is_pre_election: true
I20260807 19:31:42.579048 16506 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd2f295b92284022a3164d79b48f824a in term 1.
I20260807 19:31:42.579625 16282 leader_election.cc:302] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd2f295b92284022a3164d79b48f824a, cd37ceed4b604db5bbe24d6f070ae26d; no voters: 1eca52ce8d2d4ce4a2a2f52245054831
I20260807 19:31:42.579998 17067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:31:42.580192 17067 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1eca52ce8d2d4ce4a2a2f52245054831)
I20260807 19:31:42.580348 17067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:31:42.581501 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.572360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.8.216.189:42159)
0807 19:31:42.573407 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.573417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.573440 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:31:42.573513 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.573520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.573611 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.574904 (+ 1293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:42.575211 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.576830 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.579151 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.580493 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.580618 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.580704 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.581091 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.581188 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":393,"threads_started":1}
I20260807 19:31:42.581547 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.568175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:42159 (local address 127.0.0.1:53796)
0807 19:31:42.569299 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.569320 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:42.569458 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:42.574204 (+ 4746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:42.574212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:42.574250 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:42.574716 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.574731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.576984 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.576991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.578968 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.578984 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.580696 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.580705 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.580931 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.580964 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:31:42.581069 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:31:42.581204 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":832,"threads_started":1}
I20260807 19:31:42.582232 16308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd2f295b92284022a3164d79b48f824a" is_pre_election: true
I20260807 19:31:42.583452 17067 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } }
I20260807 19:31:42.583855 16308 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd37ceed4b604db5bbe24d6f070ae26d in current term 2: Already voted for candidate bd2f295b92284022a3164d79b48f824a in this term.
I20260807 19:31:42.584213 17067 leader_election.cc:288] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 election: Requested vote from peers 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249), cd37ceed4b604db5bbe24d6f070ae26d (127.8.216.188:34395)
I20260807 19:31:42.584699 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.584340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39286)
0807 19:31:42.584476 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.584565 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 19:31:42.584877 16483 leader_election.cc:302] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd37ceed4b604db5bbe24d6f070ae26d; no voters: 1eca52ce8d2d4ce4a2a2f52245054831, bd2f295b92284022a3164d79b48f824a
I20260807 19:31:42.585012 16506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd2f295b92284022a3164d79b48f824a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cd37ceed4b604db5bbe24d6f070ae26d"
I20260807 19:31:42.585178 17068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:31:42.586253 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.585907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39290)
0807 19:31:42.586033 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.586103 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:31:42.586462 16282 leader_election.cc:334] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249): Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
I20260807 19:31:42.587684 17068 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:31:42.589973 16506 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd2f295b92284022a3164d79b48f824a in term 2.
I20260807 19:31:42.590770 16282 leader_election.cc:302] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bd2f295b92284022a3164d79b48f824a, cd37ceed4b604db5bbe24d6f070ae26d; no voters: 1eca52ce8d2d4ce4a2a2f52245054831
I20260807 19:31:42.591089 17067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:31:42.591557 17067 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [term 2 LEADER]: Becoming Leader. State: Replica: bd2f295b92284022a3164d79b48f824a, State: Running, Role: LEADER
I20260807 19:31:42.591944 17067 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } }
I20260807 19:31:42.593927 17079 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd2f295b92284022a3164d79b48f824a. Latest consensus state: current_term: 2 leader_uuid: "bd2f295b92284022a3164d79b48f824a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:42.594408 17079 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:42.595081 17081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:42.598737 17081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:42.599570 17081 catalog_manager.cc:1259] Loaded cluster ID: ba522174f53942178942fa14e8246f0d
I20260807 19:31:42.599670 17081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:42.601141 17081 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:42.602180 17081 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Loaded TSK: 0
I20260807 19:31:42.603026 17081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:31:42.627242 16297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:42.634583 16297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:42.638972 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.638509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39300)
0807 19:31:42.638709 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.638807 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:31:42.639001 16506 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [term 2 FOLLOWER]: Refusing update from remote peer bd2f295b92284022a3164d79b48f824a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 19:31:42.639742 17079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 19:31:42.640170 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.639782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39314)
0807 19:31:42.639926 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.640003 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:31:42.640383 16282 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a -> Peer 1eca52ce8d2d4ce4a2a2f52245054831 (127.8.216.190:40249): Couldn't send request to peer 1eca52ce8d2d4ce4a2a2f52245054831. Status: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:31:42.642825 17068 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd2f295b92284022a3164d79b48f824a. Latest consensus state: current_term: 2 leader_uuid: "bd2f295b92284022a3164d79b48f824a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:42.643175 17068 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:42.644796 17079 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bd2f295b92284022a3164d79b48f824a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:42.645095 17079 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:42.645773 17068 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bd2f295b92284022a3164d79b48f824a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1eca52ce8d2d4ce4a2a2f52245054831" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 40249 } } peers { permanent_uuid: "bd2f295b92284022a3164d79b48f824a" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 42159 } attrs { promote: false } } peers { permanent_uuid: "cd37ceed4b604db5bbe24d6f070ae26d" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 34395 } attrs { promote: false } } }
I20260807 19:31:42.646107 17068 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd37ceed4b604db5bbe24d6f070ae26d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:31:42.664377 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.653658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40818)
0807 19:31:42.653856 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.653876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:42.654007 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:42.655262 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:42.655270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:42.655338 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:42.655778 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.655794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.658019 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.658028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.660884 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.660907 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.663224 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.663233 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.663534 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.663596 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 19:31:42.663683 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:31:42.663887 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:31:42.665692 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.653810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.8.216.129:34139)
0807 19:31:42.654766 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.654775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.654801 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:31:42.654905 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.654913 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.655022 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.655962 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:42.656329 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.657845 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.661146 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.662901 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.664316 (+ 1415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.664484 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.664991 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.665231 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":1377,"threads_started":1}
I20260807 19:31:42.666538 16615 tablet_service.cc:1467] Processing CreateTablet for tablet 47696da6017747dba0adfdc453cedb90 (DEFAULT_TABLE table=test-table [id=6473b95045544316aeb29865e9c4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:42.667066 16615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47696da6017747dba0adfdc453cedb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:42.671895 17091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.654917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.8.216.130:39767)
0807 19:31:42.665026 (+ 10109us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.665033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.665056 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:31:42.665140 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.665146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.665228 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.665342 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:42.666593 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.667336 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.669487 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.670677 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.670819 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.670941 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.671416 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.671568 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9962,"thread_start_us":10524,"threads_started":1}
I20260807 19:31:42.672200 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.654496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:39767 (local address 127.0.0.1:60020)
0807 19:31:42.655003 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.655021 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:42.655130 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:42.666038 (+ 10908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:42.666043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:42.666072 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:42.666401 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.666410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.667483 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.667490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.669263 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.669278 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.671041 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.671048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.671222 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.671259 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:31:42.671364 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:31:42.671537 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":2,"thread_start_us":584,"threads_started":1}
I20260807 19:31:42.672425 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.654085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39819 (local address 127.0.0.1:40116)
0807 19:31:42.654414 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.654432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:42.654522 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:42.666276 (+ 11754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:42.666281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:42.666312 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:42.666503 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.666515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.668045 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.668051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.669956 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:42.669972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.671529 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.671535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:42.671747 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.671779 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:31:42.671867 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:31:42.672041 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":55}
I20260807 19:31:42.673292 16770 tablet_service.cc:1467] Processing CreateTablet for tablet 47696da6017747dba0adfdc453cedb90 (DEFAULT_TABLE table=test-table [id=6473b95045544316aeb29865e9c4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:42.673544 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.654350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.8.216.131:39819)
0807 19:31:42.665919 (+ 11569us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.665928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.665951 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:31:42.666015 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.666020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.666104 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.666657 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:42.666833 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.667877 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.670153 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.671358 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.672313 (+ 955us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.672423 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.672474 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.672615 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11412,"thread_start_us":11301,"threads_started":1}
I20260807 19:31:42.673751 17092 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Bootstrap starting.
I20260807 19:31:42.673784 16770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47696da6017747dba0adfdc453cedb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:42.674889 16919 tablet_service.cc:1467] Processing CreateTablet for tablet 47696da6017747dba0adfdc453cedb90 (DEFAULT_TABLE table=test-table [id=6473b95045544316aeb29865e9c4b49a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:42.675325 16919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47696da6017747dba0adfdc453cedb90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:42.675364 17092 tablet_bootstrap.cc:654] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:42.681566 17092 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: No bootstrap required, opened a new log
I20260807 19:31:42.681751 17092 ts_tablet_manager.cc:1397] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:31:42.682206 17095 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Bootstrap starting.
I20260807 19:31:42.682356 17094 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Bootstrap starting.
I20260807 19:31:42.682623 17092 raft_consensus.cc:348] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.682842 17092 raft_consensus.cc:374] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:42.682969 17092 raft_consensus.cc:729] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b2c05ea57242c193e09d98ec58c18a, State: Initialized, Role: FOLLOWER
I20260807 19:31:42.683225 17092 consensus_queue.cc:260] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.684629 17094 tablet_bootstrap.cc:654] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:42.686575 17095 tablet_bootstrap.cc:654] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:42.688247 17094 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: No bootstrap required, opened a new log
I20260807 19:31:42.688426 17094 ts_tablet_manager.cc:1397] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:31:42.688908 17092 ts_tablet_manager.cc:1428] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 19:31:42.689435 17095 tablet_bootstrap.cc:492] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: No bootstrap required, opened a new log
I20260807 19:31:42.689591 17095 ts_tablet_manager.cc:1397] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 19:31:42.689699 17094 raft_consensus.cc:348] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.689981 17094 raft_consensus.cc:374] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:42.690083 17094 raft_consensus.cc:729] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0ecec2bb45417d9f28252354030dc6, State: Initialized, Role: FOLLOWER
I20260807 19:31:42.690336 17094 consensus_queue.cc:260] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.690464 17095 raft_consensus.cc:348] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.690699 17095 raft_consensus.cc:374] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:42.690825 17095 raft_consensus.cc:729] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6a28a6844f4794a7831fba938ae07e, State: Initialized, Role: FOLLOWER
I20260807 19:31:42.691087 17095 consensus_queue.cc:260] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.691253 17094 ts_tablet_manager.cc:1428] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:42.693029 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.691540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:33495)
0807 19:31:42.692591 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.692761 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":319,"threads_started":1}
I20260807 19:31:42.698069 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.697020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:47543)
0807 19:31:42.697656 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.697815 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":350,"threads_started":1}
I20260807 19:31:42.700076 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.699616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:36549)
0807 19:31:42.699834 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.699920 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 19:31:42.700862 16839 heartbeater.cc:503] Master 127.8.216.189:42159 was elected leader, sending a full tablet report...
I20260807 19:31:42.702016 17095 ts_tablet_manager.cc:1428] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Time spent starting tablet: real 0.012s user 0.002s sys 0.005s
I20260807 19:31:42.703321 16989 heartbeater.cc:503] Master 127.8.216.189:42159 was elected leader, sending a full tablet report...
I20260807 19:31:42.704049 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.703575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:45715)
0807 19:31:42.703790 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.703895 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 19:31:42.704665 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.692508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:49977)
0807 19:31:42.704182 (+ 11674us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.704370 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11502,"threads_started":1}
I20260807 19:31:42.706007 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.705605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:47745)
0807 19:31:42.705784 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.705880 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:31:42.925069 17097 raft_consensus.cc:482] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:42.925313 17097 raft_consensus.cc:504] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.926225 17097 leader_election.cc:288] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819), 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:42.926823 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47696da6017747dba0adfdc453cedb90" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6a28a6844f4794a7831fba938ae07e" is_pre_election: true
I20260807 19:31:42.926827 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47696da6017747dba0adfdc453cedb90" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0ecec2bb45417d9f28252354030dc6" is_pre_election: true
I20260807 19:31:42.927091 16940 raft_consensus.cc:2454] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 0.
I20260807 19:31:42.927101 16792 raft_consensus.cc:2454] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 0.
I20260807 19:31:42.927603 16565 leader_election.cc:302] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6a28a6844f4794a7831fba938ae07e, d7b2c05ea57242c193e09d98ec58c18a; no voters:
I20260807 19:31:42.927929 17097 raft_consensus.cc:2781] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:42.928061 17097 raft_consensus.cc:482] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:42.928154 17097 raft_consensus.cc:3037] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:42.930601 17097 raft_consensus.cc:504] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.931304 17097 leader_election.cc:288] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819), 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767)
I20260807 19:31:42.931857 16940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47696da6017747dba0adfdc453cedb90" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6a28a6844f4794a7831fba938ae07e"
I20260807 19:31:42.931891 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47696da6017747dba0adfdc453cedb90" candidate_uuid: "d7b2c05ea57242c193e09d98ec58c18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0ecec2bb45417d9f28252354030dc6"
I20260807 19:31:42.932099 16940 raft_consensus.cc:3037] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:42.932103 16792 raft_consensus.cc:3037] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:42.935209 16940 raft_consensus.cc:2454] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 1.
I20260807 19:31:42.935209 16792 raft_consensus.cc:2454] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7b2c05ea57242c193e09d98ec58c18a in term 1.
I20260807 19:31:42.935703 16568 leader_election.cc:302] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f0ecec2bb45417d9f28252354030dc6, 6a6a28a6844f4794a7831fba938ae07e, d7b2c05ea57242c193e09d98ec58c18a; no voters:
I20260807 19:31:42.936180 17097 raft_consensus.cc:2781] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:42.936662 17097 raft_consensus.cc:686] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 1 LEADER]: Becoming Leader. State: Replica: d7b2c05ea57242c193e09d98ec58c18a, State: Running, Role: LEADER
I20260807 19:31:42.936983 16686 heartbeater.cc:503] Master 127.8.216.189:42159 was elected leader, sending a full tablet report...
I20260807 19:31:42.937191 17097 consensus_queue.cc:237] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } }
I20260807 19:31:42.937896 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.937381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:60447)
0807 19:31:42.937569 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.937661 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:31:42.940321 16298 catalog_manager.cc:5495] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a reported cstate change: term changed from 0 to 1, leader changed from <none> to d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129). New cstate: current_term: 1 leader_uuid: "d7b2c05ea57242c193e09d98ec58c18a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7b2c05ea57242c193e09d98ec58c18a" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 } health_report { overall_health: UNKNOWN } } }
I20260807 19:31:42.961658 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.956606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.8.216.129:34139)
0807 19:31:42.956816 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.956824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:42.956851 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:31:42.957108 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:42.957114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:42.957201 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:42.957323 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:42.957760 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.958511 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.959794 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:42.960348 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:42.960494 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:42.960920 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:42.961025 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:42.961123 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:42.961291 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 19:31:42.961375 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:31:42.964823 16940 raft_consensus.cc:1261] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Refusing update from remote peer d7b2c05ea57242c193e09d98ec58c18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:42.964879 16792 raft_consensus.cc:1261] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Refusing update from remote peer d7b2c05ea57242c193e09d98ec58c18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:42.965658 17097 consensus_queue.cc:1035] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6a28a6844f4794a7831fba938ae07e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:42.966209 17105 consensus_queue.cc:1035] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0ecec2bb45417d9f28252354030dc6" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:42.968315 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.967597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:52903)
0807 19:31:42.967891 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.968041 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260807 19:31:42.970989 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.970343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:34009)
0807 19:31:42.970560 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.970733 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 19:31:42.974699 17110 mvcc.cc:205] Tried to move back new op lower bound from 7315992997737701376 to 7315992997633380352. Current Snapshot: MvccSnapshot[applied={T|T < 7315992997737701376}]
I20260807 19:31:42.976014 17108 mvcc.cc:205] Tried to move back new op lower bound from 7315992997737701376 to 7315992997633380352. Current Snapshot: MvccSnapshot[applied={T|T < 7315992997737701376 or (T in {7315992997737701376})}]
I20260807 19:31:42.976706 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.976283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.130:49501)
0807 19:31:42.976450 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.976546 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 19:31:42.981312 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.980738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:40389)
0807 19:31:42.981027 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.981118 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260807 19:31:42.983660 16298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:42.983954 16298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:42.991020 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.990163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.129:36845)
0807 19:31:42.990707 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.990831 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401}
I20260807 19:31:42.992695 17109 mvcc.cc:205] Tried to move back new op lower bound from 7315992997737701376 to 7315992997633380352. Current Snapshot: MvccSnapshot[applied={T|T < 7315992997737701376}]
I20260807 19:31:42.993119 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.991825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.8.216.131:60795)
0807 19:31:42.992828 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:42.992929 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":895}
I20260807 19:31:42.994647 16298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd2f295b92284022a3164d79b48f824a: Sending DeleteTablet for 3 replicas of tablet 47696da6017747dba0adfdc453cedb90
I20260807 19:31:42.996166 16615 tablet_service.cc:1514] Processing DeleteTablet for tablet 47696da6017747dba0adfdc453cedb90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:42 UTC) from {username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:40818
I20260807 19:31:42.996182 16770 tablet_service.cc:1514] Processing DeleteTablet for tablet 47696da6017747dba0adfdc453cedb90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:42 UTC) from {username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:60020
I20260807 19:31:42.996292 16919 tablet_service.cc:1514] Processing DeleteTablet for tablet 47696da6017747dba0adfdc453cedb90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:42 UTC) from {username='oryx', principal='oryx/127.8.216.189@KRBTEST.COM'} at 127.0.0.1:40116
I20260807 19:31:42.998742 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16464
I20260807 19:31:43.011313 17119 tablet_replica.cc:330] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: stopping tablet replica
I20260807 19:31:43.011639 17119 raft_consensus.cc:2229] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:43.012049 17119 raft_consensus.cc:2258] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:43.012879 17120 tablet_replica.cc:330] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: stopping tablet replica
I20260807 19:31:43.013198 17120 raft_consensus.cc:2229] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:43.013610 17120 raft_consensus.cc:2258] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:43.014369 17118 tablet_replica.cc:330] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: stopping tablet replica
I20260807 19:31:43.014590 17118 raft_consensus.cc:2229] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:43.015025 17120 ts_tablet_manager.cc:1905] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:43.015081 17118 raft_consensus.cc:2258] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:43.016213 17118 ts_tablet_manager.cc:1905] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:43.016268 17119 ts_tablet_manager.cc:1905] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:43.022951 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:42.999592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.8.216.129:34139)
0807 19:31:42.999788 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:42.999794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:43.000484 (+ 690us) server_negotiation.cc:407] Connection header received
0807 19:31:43.000826 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:43.000836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:43.000956 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:43.001046 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:43.003727 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.005427 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.008064 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.009064 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.009198 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:43.013688 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:43.013740 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:43.013766 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:43.013825 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:43.018454 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.019473 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.019484 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.019488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.019671 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.020764 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.020770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.020774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.021321 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:43.021627 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:43.022312 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 19:31:43.022580 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:31:43.023540 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.023002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39324)
0807 19:31:43.023288 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.023383 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260807 19:31:43.023525 17119 ts_tablet_manager.cc:1918] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:43.023665 17119 log.cc:1196] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/47696da6017747dba0adfdc453cedb90
I20260807 19:31:43.024271 17119 ts_tablet_manager.cc:1939] T 47696da6017747dba0adfdc453cedb90 P 1f0ecec2bb45417d9f28252354030dc6: Deleting consensus metadata
I20260807 19:31:43.024529 16678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:43.025091 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.024489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:39334)
0807 19:31:43.024748 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.024876 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 19:31:43.025560 16282 catalog_manager.cc:4841] TS 1f0ecec2bb45417d9f28252354030dc6 (127.8.216.130:39767): tablet 47696da6017747dba0adfdc453cedb90 (table test-table [id=6473b95045544316aeb29865e9c4b49a]) successfully deleted
I20260807 19:31:43.026993 17120 ts_tablet_manager.cc:1918] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:43.027201 17120 log.cc:1196] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/47696da6017747dba0adfdc453cedb90
I20260807 19:31:43.027949 17120 ts_tablet_manager.cc:1939] T 47696da6017747dba0adfdc453cedb90 P 6a6a28a6844f4794a7831fba938ae07e: Deleting consensus metadata
I20260807 19:31:43.029424 16280 catalog_manager.cc:4841] TS 6a6a28a6844f4794a7831fba938ae07e (127.8.216.131:39819): tablet 47696da6017747dba0adfdc453cedb90 (table test-table [id=6473b95045544316aeb29865e9c4b49a]) successfully deleted
I20260807 19:31:43.151923 17118 ts_tablet_manager.cc:1918] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:43.152235 17118 log.cc:1196] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/47696da6017747dba0adfdc453cedb90
I20260807 19:31:43.152705 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16464
I20260807 19:31:43.152737 17118 ts_tablet_manager.cc:1939] T 47696da6017747dba0adfdc453cedb90 P d7b2c05ea57242c193e09d98ec58c18a: Deleting consensus metadata
I20260807 19:31:43.181030 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16696
I20260807 19:31:43.181526 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.181028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40840)
0807 19:31:43.181245 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.181345 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 19:31:43.181959 16283 catalog_manager.cc:4584] TS d7b2c05ea57242c193e09d98ec58c18a (127.8.216.129:34139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47696da6017747dba0adfdc453cedb90: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
I20260807 19:31:43.194599 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.181905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60036 (local address 127.8.216.130:39767)
0807 19:31:43.182475 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:43.182480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:43.182496 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:43.182554 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:43.182557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:43.182623 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:43.182715 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:43.183997 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.184761 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.185951 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.186361 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.186454 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:43.188960 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:43.188976 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:43.188992 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:43.189025 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:43.191880 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.192620 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.192627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.192630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.192757 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.193165 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.193168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.193170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.193557 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:43.193774 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:43.194026 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 19:31:43.194204 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":287,"threads_started":1}
I20260807 19:31:43.195369 16832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:43.242565 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.242112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40842)
0807 19:31:43.242302 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.242400 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:31:43.243701 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.243284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40858)
0807 19:31:43.243455 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.243549 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:31:43.291283 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16696
I20260807 19:31:43.311049 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16852
I20260807 19:31:43.312358 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.311793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40864)
0807 19:31:43.312014 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.312092 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:31:43.313405 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.313072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40870)
0807 19:31:43.313221 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.313289 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 19:31:43.325109 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.311919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.8.216.131:39819)
0807 19:31:43.312738 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:43.312743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:43.312759 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:31:43.312812 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:43.312815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:43.312877 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:43.312966 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:43.314514 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.315214 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.316612 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:43.317021 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:43.317116 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:43.319530 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:43.319545 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:43.319562 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:43.319595 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:43.322488 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.323099 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.323106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.323108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.323237 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:43.323565 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:43.323568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:43.323570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:43.323931 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:43.324235 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:43.324534 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 19:31:43.324715 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":360,"threads_started":1}
I20260807 19:31:43.325811 16980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:43.415318 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.414795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40652)
0807 19:31:43.415039 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.415134 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 19:31:43.416559 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.416219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34139 (local address 127.0.0.1:40658)
0807 19:31:43.416357 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.416447 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:31:43.421144 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16852
I20260807 19:31:43.441758 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16153
I20260807 19:31:43.442950 16328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:43.568740 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:43.568284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:40249 (local address 127.0.0.1:49876)
0807 19:31:43.568472 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:43.568565 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:40249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:31:43.569419 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16153
I20260807 19:31:43.594007 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 16258
I20260807 19:31:43.595114 16527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:43.717005 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 16258
2026-08-07T19:31:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10741 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:43 dist-test-slave-8w9v krb5kdc[17132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:43 dist-test-slave-8w9v krb5kdc[17132](info): set up 2 sockets
Aug 07 19:31:43 dist-test-slave-8w9v krb5kdc[17132](info): commencing operation
krb5kdc: starting...
W20260807 19:31:44.883644 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.008s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:45 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab.
Entry for principal oryx/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab.
I20260807 19:31:45.148641 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:32801
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab
--principal=oryx/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:32801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:45.495415 17145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:45.495736 17145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:45.495924 17145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:45.503521 17145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:45.503647 17145 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:31:45.503690 17145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:45.503744 17145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:45.503849 17145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:45.503976 17145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:45.515096 17145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:32801
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.190.keytab
--principal=oryx/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:45.516595 17145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:45.517997 17145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:45.530835 17152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:45.531122 17151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:45.530834 17154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:45.531944 17145 server_base.cc:1034] running on GCE node
I20260807 19:31:45.533193 17145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:45.535431 17145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:45.536772 17145 hybrid_clock.cc:648] HybridClock initialized: now 1786131105536649 us; error 68 us; skew 500 ppm
I20260807 19:31:45.537227 17145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:45 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131105, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:45.541381 17145 init.cc:377] Logged in from keytab as oryx/127.8.216.190@KRBTEST.COM (short username oryx)
I20260807 19:31:45.543103 17145 webserver.cc:460] Webserver started at http://127.8.216.190:36859/ using document root <none> and password file <none>
I20260807 19:31:45.543889 17145 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:45.544000 17145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:45.544412 17145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:45.547725 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "2a5f06967e544d209487c6cc2fa28090"
format_stamp: "Formatted at 2026-08-07 19:31:45 on dist-test-slave-8w9v"
server_key: "01f447c466128ad467c1b9b9882b17fc"
server_key_iv: "f495a0e1bcbe098dc6168aeeadab7624"
server_key_version: "encryptionkey@0"
I20260807 19:31:45.548810 17145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "2a5f06967e544d209487c6cc2fa28090"
format_stamp: "Formatted at 2026-08-07 19:31:45 on dist-test-slave-8w9v"
server_key: "01f447c466128ad467c1b9b9882b17fc"
server_key_iv: "f495a0e1bcbe098dc6168aeeadab7624"
server_key_version: "encryptionkey@0"
I20260807 19:31:45.555689 17145 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:31:45.560768 17161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:45.562772 17145 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:31:45.562990 17145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "2a5f06967e544d209487c6cc2fa28090"
format_stamp: "Formatted at 2026-08-07 19:31:45 on dist-test-slave-8w9v"
server_key: "01f447c466128ad467c1b9b9882b17fc"
server_key_iv: "f495a0e1bcbe098dc6168aeeadab7624"
server_key_version: "encryptionkey@0"
I20260807 19:31:45.563274 17145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:45.591380 17145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:45.596220 17145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:45.596519 17145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:45.618542 17145 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:32801
I20260807 19:31:45.618549 17225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:32801 every 8 connection(s)
I20260807 19:31:45.620072 17145 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:45.624331 17226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:45.624883 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17145
I20260807 19:31:45.625128 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:45.625988 9058 external_mini_cluster.cc:1428] Setting key 2bde6dee4c38a0fe4deb9393a2013dd6
I20260807 19:31:45.640079 17226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Bootstrap starting.
Aug 07 19:31:45 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:45.643520 17226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:45.644721 17226 log.cc:824] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:45.647619 17226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: No bootstrap required, opened a new log
I20260807 19:31:45.650156 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:45.628258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.8.216.190:32801)
0807 19:31:45.629145 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:45.629156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:45.629211 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:31:45.630977 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:45.631000 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:45.631393 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:45.631734 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:45.633661 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:45.634980 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:45.636830 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:45.637534 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:45.637660 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:45.642407 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:45.642415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:45.642430 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:45.642475 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:45.645876 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:45.646628 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:45.646635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:45.646642 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:45.646818 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:45.647236 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:45.647240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:45.647242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:45.647845 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:45.648218 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:45.648652 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 19:31:45.649091 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":412,"threads_started":1}
I20260807 19:31:45.656201 17226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } }
I20260807 19:31:45.656589 17226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:45.656714 17226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5f06967e544d209487c6cc2fa28090, State: Initialized, Role: FOLLOWER
I20260807 19:31:45.657276 17226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } }
I20260807 19:31:45.657521 17226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:45.657662 17226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:45.657835 17226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:45.660040 17226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } }
I20260807 19:31:45.660656 17226 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a5f06967e544d209487c6cc2fa28090; no voters:
I20260807 19:31:45.661262 17226 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:45.661418 17232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:45.662971 17232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [term 1 LEADER]: Becoming Leader. State: Replica: 2a5f06967e544d209487c6cc2fa28090, State: Running, Role: LEADER
I20260807 19:31:45.663584 17226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:45.663514 17232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } }
I20260807 19:31:45.666733 17234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a5f06967e544d209487c6cc2fa28090. Latest consensus state: current_term: 1 leader_uuid: "2a5f06967e544d209487c6cc2fa28090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } } }
I20260807 19:31:45.667171 17234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:45.667714 17233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a5f06967e544d209487c6cc2fa28090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a5f06967e544d209487c6cc2fa28090" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 32801 } } }
I20260807 19:31:45.668032 17233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090 [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:45.670172 17241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:45.676014 17241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:45.686264 17241 catalog_manager.cc:1347] Generated new cluster ID: 6829299398b94a04b439756ed39b2d87
I20260807 19:31:45.686394 17241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:45.713465 17241 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:45.715500 17241 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:45.727295 17241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Generated new TSK 0
I20260807 19:31:45.728488 17241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
I20260807 19:31:45.796209 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:32801
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:46.146982 17254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:46.147285 17254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:46.147480 17254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:46.155051 17254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:46.155174 17254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:46.155318 17254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:31:46.155465 17254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:46.165946 17254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:46.167352 17254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:46.168866 17254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:46.182337 17259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:46.182415 17262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:46.182384 17260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:46.183688 17254 server_base.cc:1034] running on GCE node
I20260807 19:31:46.184444 17254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:46.185410 17254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:46.186682 17254 hybrid_clock.cc:648] HybridClock initialized: now 1786131106186613 us; error 39 us; skew 500 ppm
I20260807 19:31:46.187008 17254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:46 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131106, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:46.193555 17254 init.cc:377] Logged in from keytab as oryx/127.8.216.129@KRBTEST.COM (short username oryx)
I20260807 19:31:46.195560 17254 webserver.cc:460] Webserver started at http://127.8.216.129:33411/ using document root <none> and password file <none>
I20260807 19:31:46.196502 17254 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:46.196684 17254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:46.197094 17254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:46.200456 17254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "0657df2813084c4fb68a623d79a789e4"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "7166210f19b4b5f8d2bae90cb854cde4"
server_key_iv: "b7431592c3a4d51b6cd98428f7e95325"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.201283 17254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "0657df2813084c4fb68a623d79a789e4"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "7166210f19b4b5f8d2bae90cb854cde4"
server_key_iv: "b7431592c3a4d51b6cd98428f7e95325"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.206971 17254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:31:46.210716 17269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:46.212154 17254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:31:46.212358 17254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "0657df2813084c4fb68a623d79a789e4"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "7166210f19b4b5f8d2bae90cb854cde4"
server_key_iv: "b7431592c3a4d51b6cd98428f7e95325"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.212610 17254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:46.225736 17254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:46.230427 17254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:46.230734 17254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:46.233732 17254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:46.233867 17254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:46.234028 17254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:46.234110 17254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:31:46 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131106, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:46.259717 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.234133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:39761 (local address 127.8.216.190:32801)
0807 19:31:46.234741 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:46.234748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:46.236163 (+ 1415us) server_negotiation.cc:407] Connection header received
0807 19:31:46.239476 (+ 3313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:46.239484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:46.239601 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:46.239725 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:46.242618 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.243740 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.246362 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.246799 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.246989 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.252751 (+ 5762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:46.252761 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:46.252766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:46.252821 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:46.255586 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.256723 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.256728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.256730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.256906 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.257691 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.257699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.257704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.258040 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:46.258220 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:46.259079 (+ 859us) server_negotiation.cc:299] Negotiation successful
0807 19:31:46.259336 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":269,"threads_started":1}
I20260807 19:31:46.261400 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.234925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.129:39761)
0807 19:31:46.235912 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:46.235979 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:46.238975 (+ 2996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:46.240208 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:46.240222 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:46.241213 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:46.242315 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.242354 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.243906 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.243911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.246166 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.246196 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.247165 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.247177 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.247607 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.248661 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:46.248706 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:46.252452 (+ 3746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:46.255760 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.255770 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.255789 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.256556 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.257079 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.257091 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.257098 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.257493 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.258203 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:46.258212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:46.258651 (+ 439us) client_negotiation.cc:769] Sending connection context
0807 19:31:46.258945 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 19:31:46.259564 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":383,"threads_started":1}
I20260807 19:31:46.287460 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.269569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.129:43897)
0807 19:31:46.270401 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:46.270430 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:46.270648 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:46.271102 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:46.271110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:46.271961 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:46.272574 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.272596 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.274160 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.274166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.276552 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.276573 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.277940 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.277947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.278074 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.278791 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:46.278810 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:46.279647 (+ 837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:46.284518 (+ 4871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.284525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.284531 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.285084 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.285565 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.285569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.285571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.285711 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.286469 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:46.286474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:46.286686 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 19:31:46.286823 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:31:46.287114 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":401,"threads_started":1}
I20260807 19:31:46.287943 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.269266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:43897 (local address 127.8.216.190:32801)
0807 19:31:46.269467 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:46.269477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:46.270552 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 19:31:46.270821 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:46.270825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:46.270926 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:46.271025 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:46.272808 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.273967 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.276812 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.277664 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.277828 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.279909 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:46.279918 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:46.279926 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:46.279979 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:46.284308 (+ 4329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.285233 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.285240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.285243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.285427 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.285840 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.285845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.285848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.286322 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:46.286515 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:46.287014 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 19:31:46.287438 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:31:46.298728 17254 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:41213
I20260807 19:31:46.298740 17394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:41213 every 8 connection(s)
I20260807 19:31:46.300107 17254 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:46.302626 17395 heartbeater.cc:347] Connected to a master server at 127.8.216.190:32801
I20260807 19:31:46.302872 17395 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:46.303553 17395 heartbeater.cc:511] Master 127.8.216.190:32801 requested a full tablet report, sending...
I20260807 19:31:46.304194 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17254
I20260807 19:31:46.304455 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:46.305290 9058 external_mini_cluster.cc:1428] Setting key 5b4c0b25339e9fd2f890c326927ee7ce
I20260807 19:31:46.306236 17179 ts_manager.cc:194] Registered new tserver with Master: 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213)
I20260807 19:31:46.307945 17179 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:39761
WARNING: no policy specified for oryx/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
I20260807 19:31:46.363754 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:32801
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:46.706525 17399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:46.706849 17399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:46.707016 17399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:46.714483 17399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:46.714644 17399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:46.714780 17399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:31:46.714912 17399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:46.725296 17399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:46.726663 17399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:46.728259 17399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:46.741964 17407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:46.742043 17405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:46.742017 17404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:46.748204 17399 server_base.cc:1034] running on GCE node
I20260807 19:31:46.748919 17399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:46.749938 17399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:46.751174 17399 hybrid_clock.cc:648] HybridClock initialized: now 1786131106751101 us; error 62 us; skew 500 ppm
I20260807 19:31:46.751531 17399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:46 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131106, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:46.754760 17399 init.cc:377] Logged in from keytab as oryx/127.8.216.130@KRBTEST.COM (short username oryx)
I20260807 19:31:46.757171 17399 webserver.cc:460] Webserver started at http://127.8.216.130:37511/ using document root <none> and password file <none>
I20260807 19:31:46.757917 17399 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:46.758047 17399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:46.758388 17399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:46.761492 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "30c4146de56d425094711fce3cff52de"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "ce85d6c8f25f7f78f195fb1ff4817f06"
server_key_iv: "e87c01079c2f1202e308edd2e540c333"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.762293 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "30c4146de56d425094711fce3cff52de"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "ce85d6c8f25f7f78f195fb1ff4817f06"
server_key_iv: "e87c01079c2f1202e308edd2e540c333"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.767920 17399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:31:46.771510 17417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:46.772887 17399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:46.773064 17399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "30c4146de56d425094711fce3cff52de"
format_stamp: "Formatted at 2026-08-07 19:31:46 on dist-test-slave-8w9v"
server_key: "ce85d6c8f25f7f78f195fb1ff4817f06"
server_key_iv: "e87c01079c2f1202e308edd2e540c333"
server_key_version: "encryptionkey@0"
I20260807 19:31:46.773301 17399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:46.791627 17399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:46.795948 17399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:46.796320 17399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:46.799252 17399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:46.799371 17399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:46.799552 17399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:46.799635 17399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:31:46 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131106, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:46.826009 17426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.799759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:42581 (local address 127.8.216.190:32801)
0807 19:31:46.800629 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:46.800636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:46.801676 (+ 1040us) server_negotiation.cc:407] Connection header received
0807 19:31:46.804907 (+ 3231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:46.804913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:46.805028 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:46.805178 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:46.807609 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.809063 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.811603 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.812293 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.812458 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.818329 (+ 5871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:46.818334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:46.818338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:46.818376 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:46.822019 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.823078 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.823084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.823088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.823275 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.824003 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.824011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.824015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.824380 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:46.824549 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:46.825341 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 19:31:46.825602 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":611,"threads_started":1}
I20260807 19:31:46.827270 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.800494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.130:42581)
0807 19:31:46.801476 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:46.801537 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:46.804505 (+ 2968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:46.805499 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:46.805516 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:46.806414 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:46.807333 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.807365 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.809286 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.809294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.811398 (+ 2104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.811414 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.812598 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.812606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.812859 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.813705 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:46.813747 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:46.818053 (+ 4306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:46.822218 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.822228 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.822250 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.822947 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.823435 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.823446 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.823453 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.823830 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.824537 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:46.824545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:46.824957 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 19:31:46.825247 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 19:31:46.825833 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":373,"threads_started":1}
I20260807 19:31:46.849635 17426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.834644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:54873 (local address 127.8.216.190:32801)
0807 19:31:46.834898 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:46.834908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:46.835422 (+ 514us) server_negotiation.cc:407] Connection header received
0807 19:31:46.835589 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:46.835597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:46.835704 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:46.835845 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:46.837112 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.838183 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.840168 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.840701 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.840876 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.843177 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:46.843187 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:46.843195 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:46.843247 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:46.846868 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.847553 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.847558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.847566 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.847685 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:46.848079 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:46.848085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:46.848088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:46.848462 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:46.848630 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:46.849038 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:31:46.849269 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:31:46.849754 17513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:46.834611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.130:54873)
0807 19:31:46.835324 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:46.835345 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:46.835470 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:46.835855 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:46.835858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:46.836388 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:46.836960 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.836972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.838332 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.838342 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.839980 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:46.839995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:46.841170 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:46.841175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:46.841322 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:46.842151 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:46.842168 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:46.842926 (+ 758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:46.847008 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.847013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.847018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.847420 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.847802 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:46.847806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:46.847810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:46.847947 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:46.848592 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:46.848598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:46.848784 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 19:31:46.848931 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:31:46.849269 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":394,"threads_started":1}
I20260807 19:31:46.854887 17399 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:35275
I20260807 19:31:46.854928 17547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:35275 every 8 connection(s)
I20260807 19:31:46.856107 17399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:46.858196 17548 heartbeater.cc:347] Connected to a master server at 127.8.216.190:32801
I20260807 19:31:46.858388 17548 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:46.858976 17548 heartbeater.cc:511] Master 127.8.216.190:32801 requested a full tablet report, sending...
I20260807 19:31:46.860248 17179 ts_manager.cc:194] Registered new tserver with Master: 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:46.861027 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17399
I20260807 19:31:46.861057 17179 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:42581
I20260807 19:31:46.861229 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:46.861783 9058 external_mini_cluster.cc:1428] Setting key e4affce2d8755552dbbfd135deab552c
WARNING: no policy specified for oryx/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
I20260807 19:31:46.919736 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:32801
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:47.311662 17395 heartbeater.cc:503] Master 127.8.216.190:32801 was elected leader, sending a full tablet report...
W20260807 19:31:47.327458 17552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:47.327780 17552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:47.327955 17552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:47.335487 17552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:47.335639 17552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:47.335774 17552 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:31:47.335906 17552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:47.346015 17552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.216.190:32801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:47.347364 17552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:47.348829 17552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:47.363278 17558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:47.363384 17560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:47.363328 17557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:47.364190 17552 server_base.cc:1034] running on GCE node
I20260807 19:31:47.364915 17552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:47.365887 17552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:47.367105 17552 hybrid_clock.cc:648] HybridClock initialized: now 1786131107367003 us; error 71 us; skew 500 ppm
I20260807 19:31:47.367445 17552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:47 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131107, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:47.370715 17552 init.cc:377] Logged in from keytab as oryx/127.8.216.131@KRBTEST.COM (short username oryx)
I20260807 19:31:47.372225 17552 webserver.cc:460] Webserver started at http://127.8.216.131:40263/ using document root <none> and password file <none>
I20260807 19:31:47.372943 17552 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:47.373077 17552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:47.373416 17552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:47.376621 17552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "089ba22cd5fa43aabeb41708e9a83a12"
format_stamp: "Formatted at 2026-08-07 19:31:47 on dist-test-slave-8w9v"
server_key: "b86734abf05981252c069836549fafd3"
server_key_iv: "fbc2d200526a7ea20b27910f14addb65"
server_key_version: "encryptionkey@0"
I20260807 19:31:47.377473 17552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "089ba22cd5fa43aabeb41708e9a83a12"
format_stamp: "Formatted at 2026-08-07 19:31:47 on dist-test-slave-8w9v"
server_key: "b86734abf05981252c069836549fafd3"
server_key_iv: "fbc2d200526a7ea20b27910f14addb65"
server_key_version: "encryptionkey@0"
I20260807 19:31:47.382735 17552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:31:47.386296 17568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:47.387537 17552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:47.387709 17552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "089ba22cd5fa43aabeb41708e9a83a12"
format_stamp: "Formatted at 2026-08-07 19:31:47 on dist-test-slave-8w9v"
server_key: "b86734abf05981252c069836549fafd3"
server_key_iv: "fbc2d200526a7ea20b27910f14addb65"
server_key_version: "encryptionkey@0"
I20260807 19:31:47.387931 17552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:47.403632 17552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:47.408411 17552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:47.408705 17552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:47.411383 17552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:47.411521 17552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:47.411656 17552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:47.411741 17552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:31:47 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131107, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:47.436107 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:47.412192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:51961 (local address 127.8.216.190:32801)
0807 19:31:47.412765 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:47.412770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:47.413921 (+ 1151us) server_negotiation.cc:407] Connection header received
0807 19:31:47.416285 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:47.416291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:47.416442 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:47.416572 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:47.418246 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.419389 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.422034 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.422863 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.423044 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:47.428317 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:47.428326 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:47.428333 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:47.428395 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:47.432154 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:47.433354 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:47.433386 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:47.433389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:47.433525 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:47.434116 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:47.434119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:47.434121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:47.434416 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:47.434530 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:47.435624 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0807 19:31:47.435797 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":356,"threads_started":1}
I20260807 19:31:47.438449 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:47.412852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.131:51961)
0807 19:31:47.413756 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:47.413799 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:47.415903 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:47.416820 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:47.416828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:47.417455 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:47.418064 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:47.418083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.419550 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.419559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:47.421857 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:47.421875 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.423187 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.423200 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:47.423426 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:47.424383 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:47.424422 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:47.428012 (+ 3590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:47.432320 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:47.432334 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:47.432359 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:47.433196 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:47.433643 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:47.433651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:47.433657 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:47.433994 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:47.434550 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:47.434562 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:47.435160 (+ 598us) client_negotiation.cc:769] Sending connection context
0807 19:31:47.435554 (+ 394us) client_negotiation.cc:240] Negotiation successful
0807 19:31:47.436361 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":401,"threads_started":1}
I20260807 19:31:47.460338 17552 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:40241
I20260807 19:31:47.460362 17696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:40241 every 8 connection(s)
I20260807 19:31:47.461675 17552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:47.461752 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:47.446514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:56445 (local address 127.8.216.190:32801)
0807 19:31:47.447072 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:47.447081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:47.447567 (+ 486us) server_negotiation.cc:407] Connection header received
0807 19:31:47.447776 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:47.447782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:47.447860 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:47.447962 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:47.449258 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.450209 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.451703 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.452308 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.452455 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:47.455333 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:47.455337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:47.455340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:47.455375 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:47.458614 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:47.459613 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:47.459619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:47.459623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:47.459756 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:47.460150 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:47.460158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:47.460162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:47.460470 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:47.460633 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:47.461145 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 19:31:47.461394 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260807 19:31:47.462179 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:47.446600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.8.216.131:56445)
0807 19:31:47.447446 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:47.447468 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:47.447641 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:47.447991 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:47.447996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:47.448593 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:47.449099 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:47.449115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.450360 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.450365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:47.451472 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:47.451488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.452701 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.452706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:47.452952 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:47.453736 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:47.453766 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:47.455062 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:47.458750 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:47.458760 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:47.458765 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:47.459482 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:47.459871 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:47.459878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:47.459882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:47.460019 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:47.460687 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:47.460691 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:47.460902 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 19:31:47.461047 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:31:47.461540 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":532,"threads_started":1}
I20260807 19:31:47.464373 17697 heartbeater.cc:347] Connected to a master server at 127.8.216.190:32801
I20260807 19:31:47.464627 17697 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:47.465890 17697 heartbeater.cc:511] Master 127.8.216.190:32801 requested a full tablet report, sending...
I20260807 19:31:47.467293 17179 ts_manager.cc:194] Registered new tserver with Master: 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241)
I20260807 19:31:47.467382 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17552
I20260807 19:31:47.467597 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:47.468082 9058 external_mini_cluster.cc:1428] Setting key 924d1e81da73ab0f062cb21c7eb585f9
I20260807 19:31:47.468277 17179 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:51961
I20260807 19:31:47.470992 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:31:47 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:31:47 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:47 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:31:47.863989 17548 heartbeater.cc:503] Master 127.8.216.190:32801 was elected leader, sending a full tablet report...
I20260807 19:31:48.470890 17697 heartbeater.cc:503] Master 127.8.216.190:32801 was elected leader, sending a full tablet report...
W20260807 19:31:48.635535 17707 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.112s user 0.007s sys 0.011s
W20260807 19:31:48.636909 17578 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:47.517998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.8.216.190:32801)
0807 19:31:47.518234 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:47.518240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:47.518505 (+ 265us) server_negotiation.cc:407] Connection header received
0807 19:31:47.518688 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:47.518692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:47.518760 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:47.518847 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:47.520301 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.521080 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.522317 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:47.522768 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:47.522897 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.636562 (+1113665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56924: BlockingRecv error: recv got EOF from 127.0.0.1:56924 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 19:31:48 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131107, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:48.655877 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.641125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.8.216.190:32801)
0807 19:31:48.641382 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.641390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.641618 (+ 228us) server_negotiation.cc:407] Connection header received
0807 19:31:48.641768 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.641771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.641857 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.641985 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:48.643140 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.643828 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.645549 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.645985 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.646081 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.650623 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:48.650628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:48.650631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:48.650665 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:48.653288 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:48.653931 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:48.653950 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:48.653953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:48.654072 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:48.654446 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:48.654454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:48.654457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:48.654742 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:48.655018 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.655319 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.655571 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:31:48.659885 17179 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:48.667306 17179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:48.691530 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.680971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.8.216.129:41213)
0807 19:31:48.682966 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.682974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.683720 (+ 746us) server_negotiation.cc:407] Connection header received
0807 19:31:48.683854 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.683861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.684093 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.684343 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:48.684984 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.686449 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.688578 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.690079 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.690196 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.690432 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.691058 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.691234 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1810,"thread_start_us":1648,"threads_started":1}
I20260807 19:31:48.691599 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.680257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:41213 (local address 127.0.0.1:50984)
0807 19:31:48.683426 (+ 3169us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:48.683455 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:48.683579 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:48.684330 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:48.684334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:48.684363 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:48.684780 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.684792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.686575 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.686583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.688367 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.688383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.690609 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.690613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.690748 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.690780 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:31:48.690966 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 19:31:48.691174 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3009,"thread_start_us":963,"threads_started":1}
I20260807 19:31:48.691609 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.680322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.8.216.131:40241)
0807 19:31:48.680980 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.680992 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.681768 (+ 776us) server_negotiation.cc:407] Connection header received
0807 19:31:48.682193 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.682203 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.682484 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.682674 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:48.684541 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.685999 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.689199 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.690149 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.690241 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.690460 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.691076 (+ 616us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.691248 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":822,"threads_started":1}
I20260807 19:31:48.692409 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.680099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.0.0.1:47592)
0807 19:31:48.681624 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:48.681660 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:48.681939 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:48.683020 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:48.683031 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:48.683063 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:48.683444 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.683477 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.686129 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.686135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.689011 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.689025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.690670 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.690677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.690874 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.690909 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:31:48.691820 (+ 911us) client_negotiation.cc:240] Negotiation successful
0807 19:31:48.691962 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"mutex_wait_us":55,"thread_start_us":2922,"threads_started":1}
I20260807 19:31:48.693112 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.683564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.0.0.1:44760)
0807 19:31:48.684162 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:48.684185 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:48.684321 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:48.685704 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:48.685709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:48.685732 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:48.686106 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.686117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.687987 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.687992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.690204 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.690218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.692377 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.692383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.692552 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.692582 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:31:48.692662 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:31:48.692784 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":275,"threads_started":1}
I20260807 19:31:48.693430 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.683837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.8.216.130:35275)
0807 19:31:48.684950 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.684960 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.684985 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:31:48.685084 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.685091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.685305 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.685499 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:48.686307 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.687823 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.690367 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.692011 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.692166 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.692809 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.692898 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.693057 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":1769,"threads_started":1}
I20260807 19:31:48.695655 17322 tablet_service.cc:1467] Processing CreateTablet for tablet ef79debc192f4ff0834297228bac5545 (DEFAULT_TABLE table=test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:48.696980 17322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef79debc192f4ff0834297228bac5545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:48.696798 17623 tablet_service.cc:1467] Processing CreateTablet for tablet ef79debc192f4ff0834297228bac5545 (DEFAULT_TABLE table=test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:48.696801 17472 tablet_service.cc:1467] Processing CreateTablet for tablet ef79debc192f4ff0834297228bac5545 (DEFAULT_TABLE table=test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:48.698319 17623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef79debc192f4ff0834297228bac5545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:48.698552 17472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef79debc192f4ff0834297228bac5545. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:48.710144 17724 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Bootstrap starting.
I20260807 19:31:48.713201 17726 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Bootstrap starting.
I20260807 19:31:48.715036 17724 tablet_bootstrap.cc:654] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:48.716522 17724 log.cc:824] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:48.716953 17726 tablet_bootstrap.cc:654] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:48.718169 17725 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Bootstrap starting.
I20260807 19:31:48.718370 17726 log.cc:824] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:48.720214 17724 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: No bootstrap required, opened a new log
I20260807 19:31:48.720554 17724 ts_tablet_manager.cc:1397] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260807 19:31:48.723170 17725 tablet_bootstrap.cc:654] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:48.724449 17725 log.cc:824] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:48.726225 17726 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: No bootstrap required, opened a new log
I20260807 19:31:48.726526 17726 ts_tablet_manager.cc:1397] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.006s
I20260807 19:31:48.727574 17725 tablet_bootstrap.cc:492] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: No bootstrap required, opened a new log
I20260807 19:31:48.727831 17725 ts_tablet_manager.cc:1397] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 19:31:48.728854 17724 raft_consensus.cc:348] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.729223 17724 raft_consensus.cc:374] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:48.729341 17724 raft_consensus.cc:729] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Initialized, Role: FOLLOWER
I20260807 19:31:48.730024 17724 consensus_queue.cc:260] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.731746 17724 ts_tablet_manager.cc:1428] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
I20260807 19:31:48.739272 17726 raft_consensus.cc:348] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.739272 17725 raft_consensus.cc:348] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.739749 17725 raft_consensus.cc:374] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:48.739749 17726 raft_consensus.cc:374] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:48.739866 17725 raft_consensus.cc:729] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c4146de56d425094711fce3cff52de, State: Initialized, Role: FOLLOWER
I20260807 19:31:48.739861 17726 raft_consensus.cc:729] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Initialized, Role: FOLLOWER
I20260807 19:31:48.740536 17726 consensus_queue.cc:260] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.740533 17725 consensus_queue.cc:260] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.745909 17725 ts_tablet_manager.cc:1428] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Time spent starting tablet: real 0.018s user 0.007s sys 0.011s
I20260807 19:31:48.746085 17726 ts_tablet_manager.cc:1428] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent starting tablet: real 0.019s user 0.005s sys 0.014s
W20260807 19:31:48.804201 17396 tablet.cc:2369] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:31:48.859186 17549 tablet.cc:2369] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:48.921427 17734 raft_consensus.cc:482] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:48.921752 17734 raft_consensus.cc:504] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.923427 17734 leader_election.cc:288] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:48.932720 17723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.924167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:36943 (local address 127.8.216.130:35275)
0807 19:31:48.924374 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.924382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.925084 (+ 702us) server_negotiation.cc:407] Connection header received
0807 19:31:48.925320 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.925325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.925407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.925507 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:48.926284 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.927402 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.930448 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.931685 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.931877 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.931943 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.932317 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.932471 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:31:48.932829 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.924017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.8.216.131:36943)
0807 19:31:48.924952 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:48.924977 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:48.925140 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:48.925537 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:48.925543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:48.925589 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:48.926080 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.926100 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.927544 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.927548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.930254 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.930271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.931910 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.931916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.932099 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.932179 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 19:31:48.932267 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:31:48.932366 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":207,"spinlock_wait_cycles":3840,"thread_start_us":578,"threads_started":1}
I20260807 19:31:48.933019 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.923986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:41213 (local address 127.8.216.131:33915)
0807 19:31:48.924974 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:48.924989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:48.925180 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:48.925537 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:48.925543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:48.925583 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:48.926161 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.926175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.927545 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.927549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.930254 (+ 2705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:48.930271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.932097 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.932102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:48.932312 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.932346 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:31:48.932450 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:31:48.932580 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":140,"spinlock_wait_cycles":41088,"thread_start_us":511,"threads_started":1}
I20260807 19:31:48.933516 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.924051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:33915 (local address 127.8.216.129:41213)
0807 19:31:48.924300 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.924308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.925135 (+ 827us) server_negotiation.cc:407] Connection header received
0807 19:31:48.925320 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.925325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.925414 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.925507 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:48.926299 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.927402 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.930446 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.931713 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.931864 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.931943 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.932504 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.932652 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 19:31:48.934084 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef79debc192f4ff0834297228bac5545" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de" is_pre_election: true
I20260807 19:31:48.934480 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef79debc192f4ff0834297228bac5545" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4" is_pre_election: true
I20260807 19:31:48.934629 17493 raft_consensus.cc:2454] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:48.934964 17349 raft_consensus.cc:2454] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:48.935343 17571 leader_election.cc:302] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:48.935842 17734 raft_consensus.cc:2781] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:48.936017 17734 raft_consensus.cc:482] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:48.936149 17734 raft_consensus.cc:3037] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:48.938395 17734 raft_consensus.cc:504] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.939064 17734 leader_election.cc:288] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Requested vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:48.939512 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef79debc192f4ff0834297228bac5545" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4"
I20260807 19:31:48.939558 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef79debc192f4ff0834297228bac5545" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de"
I20260807 19:31:48.939716 17349 raft_consensus.cc:3037] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:48.939756 17493 raft_consensus.cc:3037] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:48.942189 17493 raft_consensus.cc:2454] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:48.942314 17349 raft_consensus.cc:2454] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:48.942639 17571 leader_election.cc:302] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:48.943002 17734 raft_consensus.cc:2781] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:48.943460 17734 raft_consensus.cc:686] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 LEADER]: Becoming Leader. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Running, Role: LEADER
I20260807 19:31:48.943938 17734 consensus_queue.cc:237] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } }
I20260807 19:31:48.948344 17179 catalog_manager.cc:5495] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } health_report { overall_health: UNKNOWN } } }
W20260807 19:31:48.965245 17698 tablet.cc:2369] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:31:48.988464 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:48.983032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.8.216.131:40241)
0807 19:31:48.983250 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:48.983259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:48.983401 (+ 142us) server_negotiation.cc:407] Connection header received
0807 19:31:48.983615 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:48.983619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:48.983714 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:48.983847 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:48.984307 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.985053 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.986413 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:48.986815 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:48.986907 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:48.987286 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:48.987877 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:48.987987 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:48.988117 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 19:31:48.988242 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:31:48.996937 17349 raft_consensus.cc:1261] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:48.996950 17493 raft_consensus.cc:1261] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:48.997933 17737 consensus_queue.cc:1035] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:48.998320 17734 consensus_queue.cc:1035] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.027943 17179 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.028260 17179 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.032016 17179 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Sending DeleteTablet for 3 replicas of tablet ef79debc192f4ff0834297228bac5545
I20260807 19:31:49.033284 17623 tablet_service.cc:1514] Processing DeleteTablet for tablet ef79debc192f4ff0834297228bac5545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47592
I20260807 19:31:49.033362 17322 tablet_service.cc:1514] Processing DeleteTablet for tablet ef79debc192f4ff0834297228bac5545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50984
I20260807 19:31:49.033954 17472 tablet_service.cc:1514] Processing DeleteTablet for tablet ef79debc192f4ff0834297228bac5545 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44760
I20260807 19:31:49.036367 17178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:49.037452 17178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:49.045599 17754 tablet_replica.cc:330] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: stopping tablet replica
I20260807 19:31:49.047286 17321 tablet_service.cc:1467] Processing CreateTablet for tablet 79e27183e8fa4ee4938d4859de871131 (DEFAULT_TABLE table=test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.047745 17471 tablet_service.cc:1467] Processing CreateTablet for tablet 79e27183e8fa4ee4938d4859de871131 (DEFAULT_TABLE table=test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.047995 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e27183e8fa4ee4938d4859de871131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.048339 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e27183e8fa4ee4938d4859de871131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.053037 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 79e27183e8fa4ee4938d4859de871131 (DEFAULT_TABLE table=test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.053642 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79e27183e8fa4ee4938d4859de871131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.057943 17725 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Bootstrap starting.
I20260807 19:31:49.058504 17752 tablet_replica.cc:330] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: stopping tablet replica
I20260807 19:31:49.058915 17752 raft_consensus.cc:2229] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.059412 17752 raft_consensus.cc:2258] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.059671 17725 tablet_bootstrap.cc:654] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.060530 17754 raft_consensus.cc:2229] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.061022 17754 raft_consensus.cc:2258] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.061278 17724 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Bootstrap starting.
I20260807 19:31:49.062314 17752 ts_tablet_manager.cc:1905] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.063374 17724 tablet_bootstrap.cc:654] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.070305 17753 tablet_replica.cc:330] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: stopping tablet replica
I20260807 19:31:49.070799 17753 raft_consensus.cc:2229] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:49.071062 17752 ts_tablet_manager.cc:1918] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.071275 17752 log.cc:1196] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/ef79debc192f4ff0834297228bac5545
I20260807 19:31:49.071620 17753 raft_consensus.cc:2258] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.072031 17752 ts_tablet_manager.cc:1939] T ef79debc192f4ff0834297228bac5545 P 0657df2813084c4fb68a623d79a789e4: Deleting consensus metadata
I20260807 19:31:49.072222 17726 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Bootstrap starting.
I20260807 19:31:49.073988 17726 tablet_bootstrap.cc:654] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.074095 17162 catalog_manager.cc:4841] TS 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213): tablet ef79debc192f4ff0834297228bac5545 (table test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]) successfully deleted
I20260807 19:31:49.075685 17753 ts_tablet_manager.cc:1905] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.076558 17726 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: No bootstrap required, opened a new log
I20260807 19:31:49.076716 17726 ts_tablet_manager.cc:1397] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:49.077651 17726 raft_consensus.cc:348] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.077890 17726 raft_consensus.cc:374] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.078007 17726 raft_consensus.cc:729] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.078333 17726 consensus_queue.cc:260] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.078943 17754 ts_tablet_manager.cc:1905] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.079037 17726 ts_tablet_manager.cc:1428] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:31:49.079663 17724 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: No bootstrap required, opened a new log
I20260807 19:31:49.079818 17724 ts_tablet_manager.cc:1397] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20260807 19:31:49.080690 17724 raft_consensus.cc:348] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.080937 17724 raft_consensus.cc:374] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.081044 17724 raft_consensus.cc:729] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.081359 17724 consensus_queue.cc:260] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.082708 17724 ts_tablet_manager.cc:1428] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:49.083479 17725 tablet_bootstrap.cc:492] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: No bootstrap required, opened a new log
I20260807 19:31:49.083660 17725 ts_tablet_manager.cc:1397] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20260807 19:31:49.084631 17725 raft_consensus.cc:348] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.084872 17725 raft_consensus.cc:374] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.085035 17725 raft_consensus.cc:729] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c4146de56d425094711fce3cff52de, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.085422 17725 consensus_queue.cc:260] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.086201 17725 ts_tablet_manager.cc:1428] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:31:49.088341 17754 ts_tablet_manager.cc:1918] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.088558 17754 log.cc:1196] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/ef79debc192f4ff0834297228bac5545
I20260807 19:31:49.089059 17753 ts_tablet_manager.cc:1918] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.089270 17754 ts_tablet_manager.cc:1939] T ef79debc192f4ff0834297228bac5545 P 30c4146de56d425094711fce3cff52de: Deleting consensus metadata
I20260807 19:31:49.089270 17753 log.cc:1196] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/ef79debc192f4ff0834297228bac5545
I20260807 19:31:49.089865 17753 ts_tablet_manager.cc:1939] T ef79debc192f4ff0834297228bac5545 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting consensus metadata
I20260807 19:31:49.091357 17163 catalog_manager.cc:4841] TS 30c4146de56d425094711fce3cff52de (127.8.216.130:35275): tablet ef79debc192f4ff0834297228bac5545 (table test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]) successfully deleted
I20260807 19:31:49.091620 17163 catalog_manager.cc:4841] TS 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241): tablet ef79debc192f4ff0834297228bac5545 (table test-table [id=6a846ef8a0a14bff87e55ca8d2ea0595]) successfully deleted
I20260807 19:31:49.160810 17734 raft_consensus.cc:482] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:49.161043 17734 raft_consensus.cc:504] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.161840 17734 leader_election.cc:288] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:49.162457 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e27183e8fa4ee4938d4859de871131" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4" is_pre_election: true
I20260807 19:31:49.162485 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e27183e8fa4ee4938d4859de871131" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de" is_pre_election: true
I20260807 19:31:49.162750 17349 raft_consensus.cc:2454] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:49.162750 17493 raft_consensus.cc:2454] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:49.163210 17571 leader_election.cc:302] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:49.163533 17734 raft_consensus.cc:2781] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:49.163661 17734 raft_consensus.cc:482] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:49.163790 17734 raft_consensus.cc:3037] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.166329 17734 raft_consensus.cc:504] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.167018 17734 leader_election.cc:288] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Requested vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:49.167558 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e27183e8fa4ee4938d4859de871131" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de"
I20260807 19:31:49.167562 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79e27183e8fa4ee4938d4859de871131" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4"
I20260807 19:31:49.167766 17493 raft_consensus.cc:3037] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.167806 17349 raft_consensus.cc:3037] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.170020 17493 raft_consensus.cc:2454] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:49.170425 17571 leader_election.cc:302] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:49.170739 17734 raft_consensus.cc:2781] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:49.170950 17734 raft_consensus.cc:686] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 LEADER]: Becoming Leader. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Running, Role: LEADER
I20260807 19:31:49.170991 17349 raft_consensus.cc:2454] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:49.171236 17734 consensus_queue.cc:237] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.174045 17177 catalog_manager.cc:5495] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } health_report { overall_health: HEALTHY } } }
Aug 07 19:31:49 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131105, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.190@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:49.224684 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.210732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.8.216.190:32801)
0807 19:31:49.211216 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.211227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.211507 (+ 280us) server_negotiation.cc:407] Connection header received
0807 19:31:49.211694 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.211703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.211756 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.211858 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:49.213032 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.213770 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.214991 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.215393 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.215533 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.219127 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:49.219131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:49.219135 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:49.219175 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:49.221952 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:49.222662 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:49.222669 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:49.222673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:49.222788 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:49.223151 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:49.223157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:49.223161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:49.223450 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:49.223595 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.224009 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.224267 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":232,"threads_started":1}
I20260807 19:31:49.224747 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.210675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:32801 (local address 127.0.0.1:56934)
0807 19:31:49.211399 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.211413 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.211576 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.211894 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.211898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.212623 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:49.212834 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.212845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.213900 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.213907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.214838 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.214849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.215554 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.215559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.215736 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.216530 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:49.216548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:49.218923 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:49.222091 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:49.222106 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:49.222114 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:49.222548 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:49.222893 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:49.222898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:49.222901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:49.223048 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:49.223572 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:49.223580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:49.223763 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.223924 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.224404 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":178,"thread_start_us":494,"threads_started":1}
I20260807 19:31:49.230633 17177 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:56934:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:31:49.231705 17177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:49.239015 17321 tablet_service.cc:1467] Processing CreateTablet for tablet 8a554ef73e90496f868d153d8fc31d59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acf64dcd612c45fba19c53356c4bcc29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:31:49.239353 17471 tablet_service.cc:1467] Processing CreateTablet for tablet 8a554ef73e90496f868d153d8fc31d59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acf64dcd612c45fba19c53356c4bcc29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:31:49.239602 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a554ef73e90496f868d153d8fc31d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.239866 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a554ef73e90496f868d153d8fc31d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.245234 17725 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: Bootstrap starting.
I20260807 19:31:49.246872 17725 tablet_bootstrap.cc:654] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.246958 17724 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: Bootstrap starting.
I20260807 19:31:49.247762 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 8a554ef73e90496f868d153d8fc31d59 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=acf64dcd612c45fba19c53356c4bcc29]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:31:49.248389 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a554ef73e90496f868d153d8fc31d59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.249044 17724 tablet_bootstrap.cc:654] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.256242 17726 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: Bootstrap starting.
I20260807 19:31:49.258656 17726 tablet_bootstrap.cc:654] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.258948 17725 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: No bootstrap required, opened a new log
I20260807 19:31:49.259142 17725 ts_tablet_manager.cc:1397] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260807 19:31:49.259964 17725 raft_consensus.cc:348] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.260262 17725 raft_consensus.cc:374] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.261454 17725 raft_consensus.cc:729] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c4146de56d425094711fce3cff52de, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.261773 17725 consensus_queue.cc:260] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.262511 17725 ts_tablet_manager.cc:1428] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:49.262341 17731 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.262732 17731 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.267599 17724 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: No bootstrap required, opened a new log
I20260807 19:31:49.267812 17724 ts_tablet_manager.cc:1397] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.013s
I20260807 19:31:49.268401 17726 tablet_bootstrap.cc:492] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: No bootstrap required, opened a new log
I20260807 19:31:49.268606 17726 ts_tablet_manager.cc:1397] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.001s
I20260807 19:31:49.268702 17724 raft_consensus.cc:348] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.268976 17724 raft_consensus.cc:374] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.269059 17724 raft_consensus.cc:729] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.269346 17724 consensus_queue.cc:260] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.269403 17726 raft_consensus.cc:348] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.269670 17726 raft_consensus.cc:374] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.269757 17726 raft_consensus.cc:729] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.270072 17724 ts_tablet_manager.cc:1428] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:31:49.269995 17726 consensus_queue.cc:260] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.269852 17730 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.270272 17730 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.270417 17734 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.270735 17734 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.271261 17726 ts_tablet_manager.cc:1428] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:49.333105 17734 raft_consensus.cc:482] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:49.333341 17734 raft_consensus.cc:504] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.334112 17734 leader_election.cc:288] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:49.334697 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a554ef73e90496f868d153d8fc31d59" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4" is_pre_election: true
I20260807 19:31:49.334746 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a554ef73e90496f868d153d8fc31d59" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de" is_pre_election: true
I20260807 19:31:49.334985 17349 raft_consensus.cc:2454] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:49.335079 17493 raft_consensus.cc:2454] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 0.
I20260807 19:31:49.335474 17569 leader_election.cc:302] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0657df2813084c4fb68a623d79a789e4, 089ba22cd5fa43aabeb41708e9a83a12; no voters:
I20260807 19:31:49.335788 17734 raft_consensus.cc:2781] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:49.335906 17734 raft_consensus.cc:482] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:49.336084 17734 raft_consensus.cc:3037] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.338493 17734 raft_consensus.cc:504] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.339154 17734 leader_election.cc:288] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Requested vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 30c4146de56d425094711fce3cff52de (127.8.216.130:35275)
I20260807 19:31:49.339560 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a554ef73e90496f868d153d8fc31d59" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4"
I20260807 19:31:49.339684 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a554ef73e90496f868d153d8fc31d59" candidate_uuid: "089ba22cd5fa43aabeb41708e9a83a12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de"
I20260807 19:31:49.339846 17349 raft_consensus.cc:3037] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.339947 17493 raft_consensus.cc:3037] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.342921 17493 raft_consensus.cc:2454] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:49.342919 17349 raft_consensus.cc:2454] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089ba22cd5fa43aabeb41708e9a83a12 in term 1.
I20260807 19:31:49.343343 17569 leader_election.cc:302] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0657df2813084c4fb68a623d79a789e4, 089ba22cd5fa43aabeb41708e9a83a12; no voters:
I20260807 19:31:49.343639 17734 raft_consensus.cc:2781] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:49.343822 17734 raft_consensus.cc:686] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 LEADER]: Becoming Leader. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Running, Role: LEADER
I20260807 19:31:49.344162 17734 consensus_queue.cc:237] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.345954 17737 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: TxnStatusTablet state changed. Reason: New leader 089ba22cd5fa43aabeb41708e9a83a12. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.346302 17737 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:31:49.347024 17770 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:31:49.347200 17770 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:31:49.347615 17178 catalog_manager.cc:5495] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 reported cstate change: term changed from 0 to 1, leader changed from <none> to 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } health_report { overall_health: HEALTHY } } }
I20260807 19:31:49.366734 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.360601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.0.0.1:47614)
0807 19:31:49.361015 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.361031 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.361148 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.361588 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.361592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.361602 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:49.361739 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.361745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.363198 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.363202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.364415 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.364445 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.365514 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.365519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.365688 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.365747 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.366211 (+ 464us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.366228 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.366290 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.366406 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238}
I20260807 19:31:49.366765 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.360623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.8.216.131:40241)
0807 19:31:49.360832 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.360839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.361117 (+ 278us) server_negotiation.cc:407] Connection header received
0807 19:31:49.361322 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.361331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.361448 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.361562 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:49.361866 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.363068 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.364574 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.365316 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.365482 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.365941 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.366109 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.366216 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.366318 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.366408 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:31:49.370560 17349 raft_consensus.cc:1261] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.370653 17493 raft_consensus.cc:1261] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.371299 17737 consensus_queue.cc:1035] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.371608 17734 consensus_queue.cc:1035] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.381011 17730 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: TxnStatusTablet state changed. Reason: New leader 089ba22cd5fa43aabeb41708e9a83a12. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.381376 17730 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.384091 17746 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.384657 17746 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:31:49.384917 17740 mvcc.cc:205] Tried to move back new op lower bound from 7315993023975395328 to 7315993023876055040. Current Snapshot: MvccSnapshot[applied={T|T < 7315993023975395328 or (T in {7315993023975395328})}]
I20260807 19:31:49.386607 17730 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.386881 17730 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 0657df2813084c4fb68a623d79a789e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.390982 17731 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: TxnStatusTablet state changed. Reason: New leader 089ba22cd5fa43aabeb41708e9a83a12. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.391455 17731 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.393957 17734 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.394276 17734 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:31:49.396886 17742 mvcc.cc:205] Tried to move back new op lower bound from 7315993023975395328 to 7315993023876055040. Current Snapshot: MvccSnapshot[applied={T|T < 7315993023975395328 or (T in {7315993023975395328})}]
I20260807 19:31:49.397861 17731 tablet_replica.cc:439] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "089ba22cd5fa43aabeb41708e9a83a12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } } }
I20260807 19:31:49.398160 17731 tablet_replica.cc:442] T 8a554ef73e90496f868d153d8fc31d59 P 30c4146de56d425094711fce3cff52de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:31:49.413779 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.406783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:38943 (local address 127.8.216.131:40241)
0807 19:31:49.406969 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.406974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.407721 (+ 747us) server_negotiation.cc:407] Connection header received
0807 19:31:49.407925 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.407932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.408028 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.408164 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:49.408930 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.410133 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.411627 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.412392 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.412551 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.412975 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.413116 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.413216 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.413351 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.413434 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:31:49.413798 17776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.406847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.8.216.131:38943)
0807 19:31:49.407595 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.407615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.407783 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.408235 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.408241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.408263 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:49.408735 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.408751 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.410273 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.410277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.411468 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.411480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.412586 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.412591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.412726 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.412784 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.413222 (+ 438us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.413231 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.413291 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.413396 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":420,"threads_started":1}
I20260807 19:31:49.424453 17493 raft_consensus.cc:1261] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.424611 17348 raft_consensus.cc:1261] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Refusing update from remote peer 089ba22cd5fa43aabeb41708e9a83a12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.425012 17734 consensus_queue.cc:1035] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.425309 17737 consensus_queue.cc:1035] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.449527 17741 mvcc.cc:205] Tried to move back new op lower bound from 7315993024197533696 to 7315993023168016384. Current Snapshot: MvccSnapshot[applied={T|T < 7315993024197533696}]
I20260807 19:31:49.514626 17178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.515005 17178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.519829 17178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Sending DeleteTablet for 3 replicas of tablet 79e27183e8fa4ee4938d4859de871131
I20260807 19:31:49.520695 17321 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e27183e8fa4ee4938d4859de871131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50984
I20260807 19:31:49.521039 17471 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e27183e8fa4ee4938d4859de871131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44760
I20260807 19:31:49.521315 17752 tablet_replica.cc:330] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: stopping tablet replica
I20260807 19:31:49.521508 17752 raft_consensus.cc:2229] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.521703 17752 raft_consensus.cc:2258] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.521971 17754 tablet_replica.cc:330] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: stopping tablet replica
I20260807 19:31:49.522149 17754 raft_consensus.cc:2229] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.522331 17754 raft_consensus.cc:2258] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.523511 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet 79e27183e8fa4ee4938d4859de871131 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47592
I20260807 19:31:49.523773 17753 tablet_replica.cc:330] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: stopping tablet replica
I20260807 19:31:49.523988 17753 raft_consensus.cc:2229] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:49.524444 17753 raft_consensus.cc:2258] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.526400 17754 ts_tablet_manager.cc:1905] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.526485 17752 ts_tablet_manager.cc:1905] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.527261 17753 ts_tablet_manager.cc:1905] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.534369 17754 ts_tablet_manager.cc:1918] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:49.534534 17753 ts_tablet_manager.cc:1918] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:49.534605 17754 log.cc:1196] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/79e27183e8fa4ee4938d4859de871131
I20260807 19:31:49.534771 17753 log.cc:1196] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/79e27183e8fa4ee4938d4859de871131
I20260807 19:31:49.535178 17754 ts_tablet_manager.cc:1939] T 79e27183e8fa4ee4938d4859de871131 P 30c4146de56d425094711fce3cff52de: Deleting consensus metadata
I20260807 19:31:49.535203 17753 ts_tablet_manager.cc:1939] T 79e27183e8fa4ee4938d4859de871131 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting consensus metadata
I20260807 19:31:49.535660 17752 ts_tablet_manager.cc:1918] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:49.535885 17752 log.cc:1196] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/79e27183e8fa4ee4938d4859de871131
I20260807 19:31:49.536510 17752 ts_tablet_manager.cc:1939] T 79e27183e8fa4ee4938d4859de871131 P 0657df2813084c4fb68a623d79a789e4: Deleting consensus metadata
I20260807 19:31:49.536821 17163 catalog_manager.cc:4841] TS 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241): tablet 79e27183e8fa4ee4938d4859de871131 (table test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]) successfully deleted
I20260807 19:31:49.537171 17163 catalog_manager.cc:4841] TS 30c4146de56d425094711fce3cff52de (127.8.216.130:35275): tablet 79e27183e8fa4ee4938d4859de871131 (table test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]) successfully deleted
I20260807 19:31:49.537869 17162 catalog_manager.cc:4841] TS 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213): tablet 79e27183e8fa4ee4938d4859de871131 (table test-table [id=6c8a96ce384d4cdb95b75056d3cc260c]) successfully deleted
Aug 07 19:31:49 dist-test-slave-8w9v krb5kdc[17132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:31:49 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:49 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:31:49.588598 17764 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:49.576661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56940 (local address 127.8.216.190:32801)
0807 19:31:49.576874 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.576881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.577286 (+ 405us) server_negotiation.cc:407] Connection header received
0807 19:31:49.577471 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.577477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.577596 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.577707 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:49.579089 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.580260 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.581543 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.582149 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.582277 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.588327 (+ 6050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56940: BlockingRecv error: recv got EOF from 127.0.0.1:56940 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 19:31:49 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM
I20260807 19:31:49.607553 17764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.591764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.8.216.190:32801)
0807 19:31:49.592010 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.592018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.592515 (+ 497us) server_negotiation.cc:407] Connection header received
0807 19:31:49.592707 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.592712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.592804 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.592909 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:49.594183 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.595324 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.596650 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.597337 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.597468 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.601265 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:49.601272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:49.601276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:49.601320 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:49.604779 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:49.605568 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:49.605574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:49.605578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:49.605737 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:49.606106 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:49.606111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:49.606115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:49.606504 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:49.606794 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.606963 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.607193 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:31:49.611419 17178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:31:49.612068 17178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:49.618990 17321 tablet_service.cc:1467] Processing CreateTablet for tablet f28565c5a6cc4f9faea83869c947a70a (DEFAULT_TABLE table=test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.619190 17471 tablet_service.cc:1467] Processing CreateTablet for tablet f28565c5a6cc4f9faea83869c947a70a (DEFAULT_TABLE table=test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.619464 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28565c5a6cc4f9faea83869c947a70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.619372 17622 tablet_service.cc:1467] Processing CreateTablet for tablet f28565c5a6cc4f9faea83869c947a70a (DEFAULT_TABLE table=test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.619647 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28565c5a6cc4f9faea83869c947a70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.619801 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f28565c5a6cc4f9faea83869c947a70a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.625869 17724 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Bootstrap starting.
I20260807 19:31:49.626186 17726 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Bootstrap starting.
I20260807 19:31:49.627590 17724 tablet_bootstrap.cc:654] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.627729 17726 tablet_bootstrap.cc:654] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.629416 17725 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Bootstrap starting.
I20260807 19:31:49.631103 17724 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: No bootstrap required, opened a new log
I20260807 19:31:49.631102 17725 tablet_bootstrap.cc:654] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.631325 17724 ts_tablet_manager.cc:1397] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 19:31:49.632386 17724 raft_consensus.cc:348] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.633805 17724 raft_consensus.cc:374] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.634208 17724 raft_consensus.cc:729] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.634656 17724 consensus_queue.cc:260] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.636413 17724 ts_tablet_manager.cc:1428] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:31:49.638790 17730 raft_consensus.cc:482] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:49.639034 17730 raft_consensus.cc:504] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.640970 17725 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: No bootstrap required, opened a new log
I20260807 19:31:49.641057 17730 leader_election.cc:288] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30c4146de56d425094711fce3cff52de (127.8.216.130:35275), 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241)
I20260807 19:31:49.641135 17725 ts_tablet_manager.cc:1397] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260807 19:31:49.645750 17725 raft_consensus.cc:348] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.646162 17725 raft_consensus.cc:374] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.646255 17725 raft_consensus.cc:729] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c4146de56d425094711fce3cff52de, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.648047 17725 consensus_queue.cc:260] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.653573 17725 ts_tablet_manager.cc:1428] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260807 19:31:49.666231 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.644938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44313 (local address 127.8.216.131:40241)
0807 19:31:49.645122 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.645128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.657138 (+ 12010us) server_negotiation.cc:407] Connection header received
0807 19:31:49.657758 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.657768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.657872 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.657959 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:49.658905 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.660504 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.663277 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.664886 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.665053 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.665139 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.665714 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.665923 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:31:49.667382 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.644452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.8.216.129:44313)
0807 19:31:49.656987 (+ 12535us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.657016 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.657525 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.658237 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.658245 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.658281 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:49.658698 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.658713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.660725 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.660731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.663085 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.663101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.665353 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.665366 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.665513 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.665546 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.666752 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.666970 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12373,"thread_start_us":12718,"threads_started":1}
I20260807 19:31:49.667766 17726 tablet_bootstrap.cc:492] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: No bootstrap required, opened a new log
I20260807 19:31:49.667865 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.641552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.8.216.129:45891)
0807 19:31:49.642717 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.642738 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.642888 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.661139 (+ 18251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.661143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.661176 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:49.661674 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.661685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.663225 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.663234 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.665389 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.665406 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.667080 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.667091 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.667263 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.667297 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.667385 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.667539 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":2374,"threads_started":1}
I20260807 19:31:49.668004 17726 ts_tablet_manager.cc:1397] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.021s
I20260807 19:31:49.668653 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28565c5a6cc4f9faea83869c947a70a" candidate_uuid: "0657df2813084c4fb68a623d79a789e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089ba22cd5fa43aabeb41708e9a83a12" is_pre_election: true
I20260807 19:31:49.669023 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.641604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:45891 (local address 127.8.216.130:35275)
0807 19:31:49.660803 (+ 19199us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.660812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.660840 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:49.660922 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.660927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.660998 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.661106 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:49.661894 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.663064 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.665571 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.666818 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.668278 (+ 1460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.668359 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.668411 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.668612 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19062,"thread_start_us":19641,"threads_started":1}
I20260807 19:31:49.668874 17726 raft_consensus.cc:348] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.669128 17726 raft_consensus.cc:374] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.669276 17726 raft_consensus.cc:729] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.669528 17492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28565c5a6cc4f9faea83869c947a70a" candidate_uuid: "0657df2813084c4fb68a623d79a789e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de" is_pre_election: true
I20260807 19:31:49.669858 17492 raft_consensus.cc:2454] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0657df2813084c4fb68a623d79a789e4 in term 0.
I20260807 19:31:49.670284 17726 consensus_queue.cc:260] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.670401 17271 leader_election.cc:302] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0657df2813084c4fb68a623d79a789e4, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:49.670714 17644 raft_consensus.cc:2454] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0657df2813084c4fb68a623d79a789e4 in term 0.
I20260807 19:31:49.670969 17726 ts_tablet_manager.cc:1428] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:49.672010 17730 raft_consensus.cc:2781] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:49.672170 17730 raft_consensus.cc:482] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:49.672293 17730 raft_consensus.cc:3037] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.674779 17730 raft_consensus.cc:504] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.675455 17730 leader_election.cc:288] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [CANDIDATE]: Term 1 election: Requested vote from peers 30c4146de56d425094711fce3cff52de (127.8.216.130:35275), 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241)
I20260807 19:31:49.676252 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28565c5a6cc4f9faea83869c947a70a" candidate_uuid: "0657df2813084c4fb68a623d79a789e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089ba22cd5fa43aabeb41708e9a83a12"
I20260807 19:31:49.676244 17492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f28565c5a6cc4f9faea83869c947a70a" candidate_uuid: "0657df2813084c4fb68a623d79a789e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c4146de56d425094711fce3cff52de"
I20260807 19:31:49.676468 17492 raft_consensus.cc:3037] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.676466 17644 raft_consensus.cc:3037] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.679141 17644 raft_consensus.cc:2454] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0657df2813084c4fb68a623d79a789e4 in term 1.
I20260807 19:31:49.679550 17271 leader_election.cc:302] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0657df2813084c4fb68a623d79a789e4, 089ba22cd5fa43aabeb41708e9a83a12; no voters:
I20260807 19:31:49.679558 17492 raft_consensus.cc:2454] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0657df2813084c4fb68a623d79a789e4 in term 1.
I20260807 19:31:49.679855 17730 raft_consensus.cc:2781] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:49.680446 17730 raft_consensus.cc:686] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 1 LEADER]: Becoming Leader. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Running, Role: LEADER
I20260807 19:31:49.681071 17730 consensus_queue.cc:237] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.684907 17177 catalog_manager.cc:5495] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0657df2813084c4fb68a623d79a789e4 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "0657df2813084c4fb68a623d79a789e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } health_report { overall_health: UNKNOWN } } }
I20260807 19:31:49.713086 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.705265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.8.216.129:41213)
0807 19:31:49.705956 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.705960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.705981 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:31:49.706078 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.706082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.706165 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.706312 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:49.706749 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.707547 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.709573 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.710277 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.710424 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.711243 (+ 819us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.712268 (+ 1025us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.712427 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.712628 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.712723 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":413,"threads_started":1}
I20260807 19:31:49.717453 17644 raft_consensus.cc:1261] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Refusing update from remote peer 0657df2813084c4fb68a623d79a789e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.718354 17492 raft_consensus.cc:1261] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Refusing update from remote peer 0657df2813084c4fb68a623d79a789e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.718349 17802 consensus_queue.cc:1035] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.718941 17730 consensus_queue.cc:1035] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.751332 17178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.751621 17178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:49.762517 17178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Sending DeleteTablet for 3 replicas of tablet f28565c5a6cc4f9faea83869c947a70a
I20260807 19:31:49.763710 17321 tablet_service.cc:1514] Processing DeleteTablet for tablet f28565c5a6cc4f9faea83869c947a70a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50984
I20260807 19:31:49.763868 17471 tablet_service.cc:1514] Processing DeleteTablet for tablet f28565c5a6cc4f9faea83869c947a70a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44760
I20260807 19:31:49.764102 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet f28565c5a6cc4f9faea83869c947a70a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:49 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47592
I20260807 19:31:49.764331 17754 tablet_replica.cc:330] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: stopping tablet replica
I20260807 19:31:49.764503 17753 tablet_replica.cc:330] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: stopping tablet replica
I20260807 19:31:49.764523 17754 raft_consensus.cc:2229] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.764756 17754 raft_consensus.cc:2258] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.764729 17753 raft_consensus.cc:2229] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:49.765033 17753 raft_consensus.cc:2258] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.765229 17752 tablet_replica.cc:330] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: stopping tablet replica
I20260807 19:31:49.765447 17752 raft_consensus.cc:2229] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:49.766053 17752 raft_consensus.cc:2258] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:49.767809 17752 ts_tablet_manager.cc:1905] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.767608 17178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:31:49.767880 17753 ts_tablet_manager.cc:1905] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 19:31:49.768399 17178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:31:49.776175 17471 tablet_service.cc:1467] Processing CreateTablet for tablet 06d960d9067a424194eb37dcb89e1c23 (DEFAULT_TABLE table=test-table [id=e1dd118ce8874bd5ad280b52adab86c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.776299 17622 tablet_service.cc:1467] Processing CreateTablet for tablet 06d960d9067a424194eb37dcb89e1c23 (DEFAULT_TABLE table=test-table [id=e1dd118ce8874bd5ad280b52adab86c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.776397 17321 tablet_service.cc:1467] Processing CreateTablet for tablet 06d960d9067a424194eb37dcb89e1c23 (DEFAULT_TABLE table=test-table [id=e1dd118ce8874bd5ad280b52adab86c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:31:49.776630 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d960d9067a424194eb37dcb89e1c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.776728 17622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d960d9067a424194eb37dcb89e1c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.776789 17321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06d960d9067a424194eb37dcb89e1c23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:31:49.783965 17726 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Bootstrap starting.
I20260807 19:31:49.784871 17725 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Bootstrap starting.
I20260807 19:31:49.785724 17726 tablet_bootstrap.cc:654] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.786018 17752 ts_tablet_manager.cc:1918] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.786146 17752 log.cc:1196] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/f28565c5a6cc4f9faea83869c947a70a
I20260807 19:31:49.786582 17725 tablet_bootstrap.cc:654] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.786662 17724 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Bootstrap starting.
I20260807 19:31:49.788318 17724 tablet_bootstrap.cc:654] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:49.793949 17752 ts_tablet_manager.cc:1939] T f28565c5a6cc4f9faea83869c947a70a P 0657df2813084c4fb68a623d79a789e4: Deleting consensus metadata
I20260807 19:31:49.795066 17162 catalog_manager.cc:4841] TS 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213): tablet f28565c5a6cc4f9faea83869c947a70a (table test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]) successfully deleted
I20260807 19:31:49.798524 17754 ts_tablet_manager.cc:1905] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:49.802415 17724 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: No bootstrap required, opened a new log
I20260807 19:31:49.802615 17724 ts_tablet_manager.cc:1397] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.007s
I20260807 19:31:49.802744 17726 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: No bootstrap required, opened a new log
I20260807 19:31:49.802945 17726 ts_tablet_manager.cc:1397] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.005s
I20260807 19:31:49.803485 17724 raft_consensus.cc:348] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.803743 17724 raft_consensus.cc:374] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.803812 17724 raft_consensus.cc:729] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0657df2813084c4fb68a623d79a789e4, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.803785 17726 raft_consensus.cc:348] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.804068 17726 raft_consensus.cc:374] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.804188 17726 raft_consensus.cc:729] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089ba22cd5fa43aabeb41708e9a83a12, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.804097 17724 consensus_queue.cc:260] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.804672 17724 ts_tablet_manager.cc:1428] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 19:31:49.805313 17726 consensus_queue.cc:260] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.806306 17754 ts_tablet_manager.cc:1918] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.807958 17753 ts_tablet_manager.cc:1918] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:31:49.807569 17725 tablet_bootstrap.cc:492] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: No bootstrap required, opened a new log
I20260807 19:31:49.816156 17753 log.cc:1196] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/f28565c5a6cc4f9faea83869c947a70a
I20260807 19:31:49.816293 17725 ts_tablet_manager.cc:1397] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.001s
I20260807 19:31:49.816769 17753 ts_tablet_manager.cc:1939] T f28565c5a6cc4f9faea83869c947a70a P 089ba22cd5fa43aabeb41708e9a83a12: Deleting consensus metadata
I20260807 19:31:49.817212 17725 raft_consensus.cc:348] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.817497 17725 raft_consensus.cc:374] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:49.817584 17725 raft_consensus.cc:729] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c4146de56d425094711fce3cff52de, State: Initialized, Role: FOLLOWER
I20260807 19:31:49.817837 17725 consensus_queue.cc:260] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.807966 17726 ts_tablet_manager.cc:1428] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:31:49.818377 17725 ts_tablet_manager.cc:1428] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:31:49.818498 17754 log.cc:1196] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/f28565c5a6cc4f9faea83869c947a70a
I20260807 19:31:49.818995 17754 ts_tablet_manager.cc:1939] T f28565c5a6cc4f9faea83869c947a70a P 30c4146de56d425094711fce3cff52de: Deleting consensus metadata
I20260807 19:31:49.820797 17163 catalog_manager.cc:4841] TS 30c4146de56d425094711fce3cff52de (127.8.216.130:35275): tablet f28565c5a6cc4f9faea83869c947a70a (table test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]) successfully deleted
I20260807 19:31:49.821118 17163 catalog_manager.cc:4841] TS 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241): tablet f28565c5a6cc4f9faea83869c947a70a (table test-table [id=f1b018a9a3574015ad0a8eccd2a2cf01]) successfully deleted
I20260807 19:31:49.852679 17731 raft_consensus.cc:482] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:31:49.852941 17731 raft_consensus.cc:504] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.854851 17731 leader_election.cc:288] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241)
I20260807 19:31:49.875543 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.856063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:43841 (local address 127.8.216.131:40241)
0807 19:31:49.856281 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.856287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.867730 (+ 11443us) server_negotiation.cc:407] Connection header received
0807 19:31:49.868013 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.868019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.868143 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.868210 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:49.868989 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.870185 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.872853 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.874217 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.874360 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.874438 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.875086 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.875244 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:31:49.876498 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.855351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.8.216.130:43841)
0807 19:31:49.867619 (+ 12268us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.867638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.867831 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.868413 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.868418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.868443 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:49.868789 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.868801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.870368 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.870374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.872655 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.872671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.874655 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.874662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.874884 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.874920 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.875812 (+ 892us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.875987 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12120,"thread_start_us":11679,"threads_started":1}
I20260807 19:31:49.877153 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d960d9067a424194eb37dcb89e1c23" candidate_uuid: "30c4146de56d425094711fce3cff52de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089ba22cd5fa43aabeb41708e9a83a12" is_pre_election: true
I20260807 19:31:49.877462 17644 raft_consensus.cc:2454] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c4146de56d425094711fce3cff52de in term 0.
I20260807 19:31:49.878098 17419 leader_election.cc:302] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:49.878576 17731 raft_consensus.cc:2781] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:31:49.878715 17731 raft_consensus.cc:482] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:31:49.878813 17731 raft_consensus.cc:3037] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.882325 17731 raft_consensus.cc:504] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.883689 17644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d960d9067a424194eb37dcb89e1c23" candidate_uuid: "30c4146de56d425094711fce3cff52de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089ba22cd5fa43aabeb41708e9a83a12"
I20260807 19:31:49.883929 17644 raft_consensus.cc:3037] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.886700 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.860389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34451 (local address 127.8.216.129:41213)
0807 19:31:49.860761 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.860768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.866860 (+ 6092us) server_negotiation.cc:407] Connection header received
0807 19:31:49.867105 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.867114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.867211 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.867338 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:31:49.876931 (+ 9593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.878138 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.883980 (+ 5842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.885371 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.885512 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.885592 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.886272 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.886432 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260807 19:31:49.887217 17644 raft_consensus.cc:2454] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c4146de56d425094711fce3cff52de in term 1.
I20260807 19:31:49.887689 17419 leader_election.cc:302] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089ba22cd5fa43aabeb41708e9a83a12, 30c4146de56d425094711fce3cff52de; no voters:
I20260807 19:31:49.888161 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.855223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:41213 (local address 127.8.216.130:34451)
0807 19:31:49.866738 (+ 11515us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.866761 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.866946 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.875905 (+ 8959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.875911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.875941 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:31:49.876366 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.876377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.878513 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.878518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.880911 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.880927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.885852 (+ 4925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.885858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.886056 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.886092 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.887194 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.887405 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11371,"thread_start_us":12247,"threads_started":1}
I20260807 19:31:49.888204 17731 leader_election.cc:288] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [CANDIDATE]: Term 1 election: Requested vote from peers 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213), 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241)
I20260807 19:31:49.890513 17731 raft_consensus.cc:2781] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:49.890723 17731 raft_consensus.cc:686] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 1 LEADER]: Becoming Leader. State: Replica: 30c4146de56d425094711fce3cff52de, State: Running, Role: LEADER
I20260807 19:31:49.891371 17731 consensus_queue.cc:237] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } }
I20260807 19:31:49.893967 17348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d960d9067a424194eb37dcb89e1c23" candidate_uuid: "30c4146de56d425094711fce3cff52de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4" is_pre_election: true
I20260807 19:31:49.894085 17349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06d960d9067a424194eb37dcb89e1c23" candidate_uuid: "30c4146de56d425094711fce3cff52de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0657df2813084c4fb68a623d79a789e4"
I20260807 19:31:49.894326 17348 raft_consensus.cc:2454] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c4146de56d425094711fce3cff52de in term 0.
I20260807 19:31:49.901672 17177 catalog_manager.cc:5495] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de reported cstate change: term changed from 0 to 1, leader changed from <none> to 30c4146de56d425094711fce3cff52de (127.8.216.130). New cstate: current_term: 1 leader_uuid: "30c4146de56d425094711fce3cff52de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30c4146de56d425094711fce3cff52de" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 35275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 } health_report { overall_health: UNKNOWN } } }
I20260807 19:31:49.941713 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.933916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.8.216.130:35275)
0807 19:31:49.934099 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.934106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.934126 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:31:49.934233 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.934239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.934354 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.934518 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:49.935516 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.936731 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.938760 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.939456 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.939586 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.940247 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.941057 (+ 810us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.941208 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.941336 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.941430 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 19:31:49.961676 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.954321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:45575 (local address 127.8.216.131:40241)
0807 19:31:49.954544 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:49.954551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:49.954755 (+ 204us) server_negotiation.cc:407] Connection header received
0807 19:31:49.955067 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:49.955073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:49.955185 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:49.955293 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:49.955907 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.957161 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.959303 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.960052 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.960248 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.960793 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.960972 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.961067 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:49.961244 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 19:31:49.961340 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:31:49.961797 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:49.953945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40241 (local address 127.8.216.130:45575)
0807 19:31:49.954641 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:49.954667 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:49.954896 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:49.955334 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:49.955339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:49.955352 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:49.955729 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.955741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.957402 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.957408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.959146 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:49.959162 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:49.960304 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:49.960311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:49.960501 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:49.960580 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:49.961082 (+ 502us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:49.961094 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:31:49.961184 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 19:31:49.961338 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":622,"threads_started":1}
I20260807 19:31:49.973974 17644 raft_consensus.cc:1261] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Refusing update from remote peer 30c4146de56d425094711fce3cff52de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.974601 17348 raft_consensus.cc:3037] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:49.974642 17731 consensus_queue.cc:1035] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [LEADER]: Connected to new peer: Peer: permanent_uuid: "089ba22cd5fa43aabeb41708e9a83a12" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:49.979301 17348 raft_consensus.cc:1261] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Refusing update from remote peer 30c4146de56d425094711fce3cff52de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:31:49.979928 17731 consensus_queue.cc:1035] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [LEADER]: Connected to new peer: Peer: permanent_uuid: "0657df2813084c4fb68a623d79a789e4" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 41213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:31:50.045150 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.037551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:52913 (local address 127.8.216.130:35275)
0807 19:31:50.037770 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:50.037777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:50.037971 (+ 194us) server_negotiation.cc:407] Connection header received
0807 19:31:50.038268 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:50.038274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:50.038379 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:50.038446 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:31:50.039247 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.040467 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.042847 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.043538 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.043685 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:50.044255 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:31:50.044408 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:31:50.044502 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:50.044712 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:31:50.044804 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:31:50.045959 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.037209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.8.216.131:52913)
0807 19:31:50.037848 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:50.037878 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:50.038091 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:50.038668 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:50.038673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:50.038686 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:31:50.039045 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:50.039058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.040675 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.040680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:50.042670 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:50.042688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.043762 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.043770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:50.043980 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:50.044037 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:31:50.044562 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:31:50.044575 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 19:31:50.045355 (+ 780us) client_negotiation.cc:240] Negotiation successful
0807 19:31:50.045520 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":1736,"threads_started":1}
I20260807 19:31:50.104933 17177 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:50.105232 17177 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:31:50.109270 17177 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a5f06967e544d209487c6cc2fa28090: Sending DeleteTablet for 3 replicas of tablet 06d960d9067a424194eb37dcb89e1c23
I20260807 19:31:50.110533 17321 tablet_service.cc:1514] Processing DeleteTablet for tablet 06d960d9067a424194eb37dcb89e1c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:50 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50984
I20260807 19:31:50.110852 17752 tablet_replica.cc:330] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: stopping tablet replica
I20260807 19:31:50.111069 17752 raft_consensus.cc:2229] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:50.111186 17471 tablet_service.cc:1514] Processing DeleteTablet for tablet 06d960d9067a424194eb37dcb89e1c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:50 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44760
I20260807 19:31:50.111294 17752 raft_consensus.cc:2258] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:50.111613 17754 tablet_replica.cc:330] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: stopping tablet replica
I20260807 19:31:50.112039 17754 raft_consensus.cc:2229] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:31:50.112589 17754 raft_consensus.cc:2258] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:50.112862 17752 ts_tablet_manager.cc:1905] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:50.117063 17754 ts_tablet_manager.cc:1905] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:50.121940 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet 06d960d9067a424194eb37dcb89e1c23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:31:50 UTC) from {username='oryx', principal='oryx/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47592
I20260807 19:31:50.122325 17753 tablet_replica.cc:330] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: stopping tablet replica
I20260807 19:31:50.122524 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17254
I20260807 19:31:50.122606 17753 raft_consensus.cc:2229] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:31:50.122830 17753 raft_consensus.cc:2258] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:31:50.124657 17753 ts_tablet_manager.cc:1905] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:31:50.130404 17752 ts_tablet_manager.cc:1918] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:50.130467 17754 ts_tablet_manager.cc:1918] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:50.130591 17752 log.cc:1196] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/06d960d9067a424194eb37dcb89e1c23
I20260807 19:31:50.130811 17754 log.cc:1196] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/06d960d9067a424194eb37dcb89e1c23
I20260807 19:31:50.131063 17752 ts_tablet_manager.cc:1939] T 06d960d9067a424194eb37dcb89e1c23 P 0657df2813084c4fb68a623d79a789e4: Deleting consensus metadata
I20260807 19:31:50.131433 17754 ts_tablet_manager.cc:1939] T 06d960d9067a424194eb37dcb89e1c23 P 30c4146de56d425094711fce3cff52de: Deleting consensus metadata
I20260807 19:31:50.132443 17162 catalog_manager.cc:4841] TS 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213): tablet 06d960d9067a424194eb37dcb89e1c23 (table test-table [id=e1dd118ce8874bd5ad280b52adab86c5]) successfully deleted
I20260807 19:31:50.132728 17163 catalog_manager.cc:4841] TS 30c4146de56d425094711fce3cff52de (127.8.216.130:35275): tablet 06d960d9067a424194eb37dcb89e1c23 (table test-table [id=e1dd118ce8874bd5ad280b52adab86c5]) successfully deleted
I20260807 19:31:50.139694 17753 ts_tablet_manager.cc:1918] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:31:50.139918 17753 log.cc:1196] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/06d960d9067a424194eb37dcb89e1c23
I20260807 19:31:50.140803 17753 ts_tablet_manager.cc:1939] T 06d960d9067a424194eb37dcb89e1c23 P 089ba22cd5fa43aabeb41708e9a83a12: Deleting consensus metadata
I20260807 19:31:50.142098 17163 catalog_manager.cc:4841] TS 089ba22cd5fa43aabeb41708e9a83a12 (127.8.216.131:40241): tablet 06d960d9067a424194eb37dcb89e1c23 (table test-table [id=e1dd118ce8874bd5ad280b52adab86c5]) successfully deleted
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.129@KRBTEST.COM
I20260807 19:31:50.154203 17805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.125110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.8.216.129:41213)
0807 19:31:50.125281 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:50.125287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:50.133380 (+ 8093us) server_negotiation.cc:407] Connection header received
0807 19:31:50.133606 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:50.133611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:50.133721 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:50.133831 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:50.135646 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.136902 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.138724 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.139478 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.139610 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:50.145708 (+ 6098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:50.145745 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:50.145768 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:50.145827 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:50.150441 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.151414 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.151426 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.151429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.151611 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.152064 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.152069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.152073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.152605 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:50.152968 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:50.153559 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 19:31:50.153810 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:31:50.156072 17390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:50.292104 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17254
I20260807 19:31:50.317853 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17399
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.130@KRBTEST.COM
I20260807 19:31:50.336076 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.318984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.8.216.130:35275)
0807 19:31:50.319163 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:50.319168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:50.319186 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:31:50.319246 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:50.319250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:50.319314 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:50.319403 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:50.321045 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.321804 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.323631 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.324092 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.324465 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:50.329660 (+ 5195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:50.329675 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:50.329693 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:50.329729 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:50.333136 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.333810 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.333817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.333820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.333971 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.334360 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.334370 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.334372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.334806 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:50.335123 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:50.335453 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:31:50.335697 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:31:50.336776 17543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:50.465698 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17399
I20260807 19:31:50.481014 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.480217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:41213 (local address 127.8.216.131:50679)
0807 19:31:50.480722 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:50.480846 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:41213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":281,"threads_started":1}
W20260807 19:31:50.481406 17569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.129:41213: connect: Connection refused (error 111)
I20260807 19:31:50.483006 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.482553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:41213 (local address 127.8.216.131:49595)
0807 19:31:50.482779 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:50.482896 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:41213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 19:31:50.483306 17569 consensus_peers.cc:487] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 -> Peer 0657df2813084c4fb68a623d79a789e4 (127.8.216.129:41213): Couldn't send request to peer 0657df2813084c4fb68a623d79a789e4. Status: Network error: Client connection negotiation failed: client connection to 127.8.216.129:41213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:31:50.497417 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17552
I20260807 19:31:50.505179 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.504733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.8.216.131:49517)
0807 19:31:50.504895 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:50.504991 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.130:35275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:31:50.506297 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.505918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:35275 (local address 127.8.216.131:59479)
0807 19:31:50.506079 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:50.506167 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.130:35275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:31:50.506551 17571 consensus_peers.cc:487] T 8a554ef73e90496f868d153d8fc31d59 P 089ba22cd5fa43aabeb41708e9a83a12 -> Peer 30c4146de56d425094711fce3cff52de (127.8.216.130:35275): Couldn't send request to peer 30c4146de56d425094711fce3cff52de. Status: Network error: Client connection negotiation failed: client connection to 127.8.216.130:35275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.216.131@KRBTEST.COM
I20260807 19:31:50.515331 17833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:50.498332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47616 (local address 127.8.216.131:40241)
0807 19:31:50.498908 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:50.498913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:50.498927 (+ 14us) server_negotiation.cc:407] Connection header received
0807 19:31:50.498978 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:50.498981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:50.499037 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:50.499126 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:50.500874 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.501718 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.503584 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:50.504313 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:50.504449 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:50.509639 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:50.509657 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:50.509673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:50.509710 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:50.512499 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.513203 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.513212 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.513214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.513343 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:50.513712 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:50.513717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:50.513720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:50.514131 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:50.514435 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:50.514747 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 19:31:50.514987 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":318,"threads_started":1}
I20260807 19:31:50.515995 17692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:50.664875 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17552
I20260807 19:31:50.694896 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17145
I20260807 19:31:50.696170 17221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:50.820283 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17145
2026-08-07T19:31:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7103 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17838](info): set up 2 sockets
Aug 07 19:31:50 dist-test-slave-8w9v krb5kdc[17838](info): commencing operation
krb5kdc: starting...
W20260807 19:31:51.979404 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:52 dist-test-slave-8w9v krb5kdc[17838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:52.251724 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44599
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:37135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:52.602797 17851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:52.603106 17851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:52.603284 17851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:52.610726 17851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:52.610906 17851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:52.611033 17851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:52.611105 17851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:52.611155 17851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:52.621587 17851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:37135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:52.622906 17851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:52.624349 17851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:52.636461 17857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:52.636544 17858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:52.636502 17860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:52.637568 17851 server_base.cc:1034] running on GCE node
I20260807 19:31:52.638450 17851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:52.639950 17851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:52.641192 17851 hybrid_clock.cc:648] HybridClock initialized: now 1786131112641131 us; error 52 us; skew 500 ppm
I20260807 19:31:52.641575 17851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T19:31:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1888 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:31:52 dist-test-slave-8w9v krb5kdc[17866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:31:52 dist-test-slave-8w9v krb5kdc[17866](info): set up 2 sockets
Aug 07 19:31:52 dist-test-slave-8w9v krb5kdc[17866](info): commencing operation
krb5kdc: starting...
W20260807 19:31:53.874804 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:31:54 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:31:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:31:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:31:54.143913 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33351
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:54.488250 17879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:54.488569 17879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:54.488662 17879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:54.496627 17879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:54.496743 17879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:54.496804 17879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:54.496865 17879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:54.496908 17879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:54.507177 17879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:54.508533 17879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:54.509842 17879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:54.521944 17887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:54.521901 17888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:54.521858 17890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:54.522706 17879 server_base.cc:1034] running on GCE node
I20260807 19:31:54.523517 17879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:54.525036 17879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:54.526268 17879 hybrid_clock.cc:648] HybridClock initialized: now 1786131114526221 us; error 44 us; skew 500 ppm
I20260807 19:31:54.526612 17879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:54 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:54.529846 17879 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:54.531441 17879 webserver.cc:460] Webserver started at http://127.8.216.190:45259/ using document root <none> and password file <none>
I20260807 19:31:54.532213 17879 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:54.532362 17879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:54.532687 17879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:54.535773 17879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "52bd920b59494593b689f58a6b0ccd7d"
format_stamp: "Formatted at 2026-08-07 19:31:54 on dist-test-slave-8w9v"
server_key: "3bedb1faf47ebafb57d09389cc1474ec"
server_key_iv: "c4edbc8958175b1e40f301e1786e6bcd"
server_key_version: "encryptionkey@0"
I20260807 19:31:54.536641 17879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "52bd920b59494593b689f58a6b0ccd7d"
format_stamp: "Formatted at 2026-08-07 19:31:54 on dist-test-slave-8w9v"
server_key: "3bedb1faf47ebafb57d09389cc1474ec"
server_key_iv: "c4edbc8958175b1e40f301e1786e6bcd"
server_key_version: "encryptionkey@0"
I20260807 19:31:54.541968 17879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:54.545599 17897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:54.546900 17879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:54.547082 17879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "52bd920b59494593b689f58a6b0ccd7d"
format_stamp: "Formatted at 2026-08-07 19:31:54 on dist-test-slave-8w9v"
server_key: "3bedb1faf47ebafb57d09389cc1474ec"
server_key_iv: "c4edbc8958175b1e40f301e1786e6bcd"
server_key_version: "encryptionkey@0"
I20260807 19:31:54.547288 17879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:54.573159 17879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:54.577745 17879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:54.578056 17879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:54.597296 17879 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33351
I20260807 19:31:54.597317 17949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33351 every 8 connection(s)
I20260807 19:31:54.598793 17879 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:54.599915 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17879
I20260807 19:31:54.600193 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:31:54.600863 9058 external_mini_cluster.cc:1428] Setting key 11c79bd0de5490d17dfab9a3e63e5ec6
I20260807 19:31:54.603446 17950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:31:54 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:54.617601 17950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Bootstrap starting.
I20260807 19:31:54.622081 17950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Neither blocks nor log segments found. Creating new log.
I20260807 19:31:54.623540 17950 log.cc:824] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:54.626138 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:54.602895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57066 (local address 127.8.216.190:33351)
0807 19:31:54.603735 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:54.603747 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:54.603787 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:31:54.605588 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:54.605609 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:54.606018 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:54.606354 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:54.608337 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:54.609710 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:54.611526 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:54.612244 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:54.612388 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:54.617399 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:54.617411 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:54.617433 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:54.617489 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:54.621710 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:54.622529 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:54.622541 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:54.622552 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:54.622785 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:54.623131 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:54.623136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:54.623138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:54.623899 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:54.624301 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:54.624670 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 19:31:54.625110 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":398,"threads_started":1}
I20260807 19:31:54.626845 17950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: No bootstrap required, opened a new log
I20260807 19:31:54.634706 17950 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:54.635118 17950 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:31:54.635226 17950 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52bd920b59494593b689f58a6b0ccd7d, State: Initialized, Role: FOLLOWER
I20260807 19:31:54.635831 17950 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:54.636106 17950 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:54.636322 17950 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:54.636507 17950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:31:54.638765 17950 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:54.639317 17950 leader_election.cc:302] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52bd920b59494593b689f58a6b0ccd7d; no voters:
I20260807 19:31:54.640033 17950 leader_election.cc:288] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:31:54.640231 17955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:31:54.640940 17955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 LEADER]: Becoming Leader. State: Replica: 52bd920b59494593b689f58a6b0ccd7d, State: Running, Role: LEADER
I20260807 19:31:54.641563 17955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:54.642189 17950 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:54.644640 17958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52bd920b59494593b689f58a6b0ccd7d. Latest consensus state: current_term: 1 leader_uuid: "52bd920b59494593b689f58a6b0ccd7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } } }
I20260807 19:31:54.644747 17956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52bd920b59494593b689f58a6b0ccd7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } } }
I20260807 19:31:54.645076 17958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:54.645159 17956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:54.646085 17963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:54.652813 17963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:54.665781 17963 catalog_manager.cc:1347] Generated new cluster ID: 54e35538e0224e69993113876f519732
I20260807 19:31:54.665921 17963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:54.696180 17963 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:31:54.697882 17963 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:54.705390 17963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Generated new TSK 0
I20260807 19:31:54.706279 17963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:31:54.774035 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:55.123535 17978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:55.123849 17978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:55.123955 17978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:55.132479 17978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:55.132604 17978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:55.132725 17978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:55.142800 17978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:55.144349 17978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:55.145792 17978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:55.159622 17983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:55.159555 17984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:55.159582 17986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:55.160460 17978 server_base.cc:1034] running on GCE node
I20260807 19:31:55.161056 17978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:55.161964 17978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:55.163180 17978 hybrid_clock.cc:648] HybridClock initialized: now 1786131115163121 us; error 38 us; skew 500 ppm
I20260807 19:31:55.163537 17978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:55 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131115, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:55.166791 17978 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:31:55.168411 17978 webserver.cc:460] Webserver started at http://127.8.216.129:33941/ using document root <none> and password file <none>
I20260807 19:31:55.169070 17978 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:55.169169 17978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:55.169489 17978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:55.172484 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "7e22874e186744b1bec933f83341eb15"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "37b0959e16049a28a1e33cc5ee35085a"
server_key_iv: "7c3ac5b1bf86edcff1a5fe10a476ce80"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.173236 17978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "7e22874e186744b1bec933f83341eb15"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "37b0959e16049a28a1e33cc5ee35085a"
server_key_iv: "7c3ac5b1bf86edcff1a5fe10a476ce80"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.178418 17978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:31:55.182019 17993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.183347 17978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:55.183523 17978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "7e22874e186744b1bec933f83341eb15"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "37b0959e16049a28a1e33cc5ee35085a"
server_key_iv: "7c3ac5b1bf86edcff1a5fe10a476ce80"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.183698 17978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:55.213377 17978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:55.218114 17978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:55.218420 17978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:55.219405 17978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:55.220880 17978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:55.220971 17978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.221122 17978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:55.221189 17978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.256191 17978 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38049
I20260807 19:31:55.256249 18115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38049 every 8 connection(s)
I20260807 19:31:55.257555 17978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:55.261365 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 17978
I20260807 19:31:55.261612 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:31:55.262305 9058 external_mini_cluster.cc:1428] Setting key 1d9abfb43c2eb0028bc916efc41f2270
Aug 07 19:31:55 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131115, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:55.286023 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:55.260229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:51911 (local address 127.8.216.190:33351)
0807 19:31:55.260733 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:55.260738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:55.261766 (+ 1028us) server_negotiation.cc:407] Connection header received
0807 19:31:55.263878 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:55.263883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:55.263951 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:55.264078 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:55.265797 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.266995 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.268731 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.269426 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.269600 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:55.279480 (+ 9880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:55.279486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:55.279490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:55.279528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:55.282328 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:55.283244 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:55.283248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:55.283250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:55.283387 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:55.283881 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:55.283885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:55.283887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:55.284276 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:55.284467 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:55.285564 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0807 19:31:55.285762 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":286,"threads_started":1}
I20260807 19:31:55.287847 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:55.260735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:51911)
0807 19:31:55.261574 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:55.261621 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:55.263563 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:55.264403 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:55.264415 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:55.264993 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:55.265602 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:55.265620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.267188 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.267192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:55.268580 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:55.268593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.269678 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.269683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:55.269914 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:55.276092 (+ 6178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:55.276158 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:55.279166 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:55.282542 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:55.282554 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:55.282581 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:55.283082 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:55.283519 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:55.283525 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:55.283528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:55.283747 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:55.284484 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:55.284497 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:55.285070 (+ 573us) client_negotiation.cc:769] Sending connection context
0807 19:31:55.285456 (+ 386us) client_negotiation.cc:240] Negotiation successful
0807 19:31:55.286187 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":363,"threads_started":1}
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:31:55.289652 18116 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33351
I20260807 19:31:55.289983 18116 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:55.290827 18116 heartbeater.cc:511] Master 127.8.216.190:33351 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
I20260807 19:31:55.293146 17914 ts_manager.cc:194] Registered new tserver with Master: 7e22874e186744b1bec933f83341eb15 (127.8.216.129:38049)
I20260807 19:31:55.294731 17914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:51911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:31:55.323763 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:55.686602 18122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:55.686950 18122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:55.687103 18122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:55.694712 18122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:55.694831 18122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:55.694955 18122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:55.704821 18122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:55.706346 18122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:55.707624 18122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:55.721252 18127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:55.721192 18131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:55.721215 18128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:55.722179 18122 server_base.cc:1034] running on GCE node
I20260807 19:31:55.722877 18122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:55.723898 18122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:55.725140 18122 hybrid_clock.cc:648] HybridClock initialized: now 1786131115725086 us; error 45 us; skew 500 ppm
I20260807 19:31:55.725510 18122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:55 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131115, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:55.728708 18122 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:31:55.730118 18122 webserver.cc:460] Webserver started at http://127.8.216.130:44991/ using document root <none> and password file <none>
I20260807 19:31:55.731084 18122 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:55.731238 18122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:55.731668 18122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:55.735226 18122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "3a02b88628584681a5fe01ae157cd3e0"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "a56c3fa10a513b938987ecd75fb3564b"
server_key_iv: "1212e8d2d4605580af6ed072534a5894"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.736101 18122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "3a02b88628584681a5fe01ae157cd3e0"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "a56c3fa10a513b938987ecd75fb3564b"
server_key_iv: "1212e8d2d4605580af6ed072534a5894"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.741376 18122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:31:55.745373 18138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.747298 18122 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:31:55.747504 18122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "3a02b88628584681a5fe01ae157cd3e0"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "a56c3fa10a513b938987ecd75fb3564b"
server_key_iv: "1212e8d2d4605580af6ed072534a5894"
server_key_version: "encryptionkey@0"
I20260807 19:31:55.747764 18122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:55.775370 18122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:55.779729 18122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:55.780020 18122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:55.781071 18122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:55.782425 18122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:55.782510 18122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.782599 18122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:55.782644 18122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:55.817430 18122 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:40165
I20260807 19:31:55.817556 18258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:40165 every 8 connection(s)
I20260807 19:31:55.818787 18122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:55.821713 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18122
I20260807 19:31:55.821954 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:31:55.822731 9058 external_mini_cluster.cc:1428] Setting key 8f46158b207b11b9a3adc6fd75997c61
Aug 07 19:31:55 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131115, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:55.842139 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:55.821704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:35011 (local address 127.8.216.190:33351)
0807 19:31:55.824185 (+ 2481us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:55.824191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:55.824212 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:31:55.826581 (+ 2369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:55.826589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:55.826682 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:55.826809 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:55.828395 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.829507 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.831100 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.831842 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.831987 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:55.835682 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:55.835687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:55.835690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:55.835722 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:55.838654 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:55.839624 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:55.839627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:55.839629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:55.839759 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:55.840285 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:55.840288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:55.840290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:55.840553 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:55.840674 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:55.841603 (+ 929us) server_negotiation.cc:299] Negotiation successful
0807 19:31:55.841770 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2283,"thread_start_us":2172,"threads_started":1}
I20260807 19:31:55.843668 18262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:55.822220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:35011)
0807 19:31:55.823176 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:55.823225 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:55.826292 (+ 3067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:55.827027 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:55.827035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:55.827558 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:55.828212 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:55.828240 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.829661 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.829665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:55.830935 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:55.830961 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:55.832056 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:55.832060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:55.832340 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:55.833084 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:55.833107 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:55.835431 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:55.838800 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:55.838812 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:55.838832 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:55.839503 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:55.839865 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:55.839872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:55.839879 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:55.840177 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:55.840699 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:55.840711 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:55.841169 (+ 458us) client_negotiation.cc:769] Sending connection context
0807 19:31:55.841508 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 19:31:55.842111 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":383,"threads_started":1}
I20260807 19:31:55.845233 18259 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33351
I20260807 19:31:55.845566 18259 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:55.846398 18259 heartbeater.cc:511] Master 127.8.216.190:33351 requested a full tablet report, sending...
I20260807 19:31:55.848191 17914 ts_manager.cc:194] Registered new tserver with Master: 3a02b88628584681a5fe01ae157cd3e0 (127.8.216.130:40165)
I20260807 19:31:55.849553 17914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:35011
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:31:55.887166 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:56.230893 18265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:56.231218 18265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:56.231412 18265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:56.238686 18265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:56.238796 18265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:56.238910 18265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:56.248900 18265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:56.250198 18265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:56.251560 18265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:56.266422 18270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:56.266480 18272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:56.266642 18275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:56.267310 18265 server_base.cc:1034] running on GCE node
I20260807 19:31:56.267933 18265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:56.269065 18265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:56.270300 18265 hybrid_clock.cc:648] HybridClock initialized: now 1786131116270233 us; error 76 us; skew 500 ppm
I20260807 19:31:56.270639 18265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:56 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131116, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:56.273895 18265 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:31:56.275449 18265 webserver.cc:460] Webserver started at http://127.8.216.131:46373/ using document root <none> and password file <none>
I20260807 19:31:56.276250 18265 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:56.276400 18265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:56.276747 18265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:31:56.279733 18265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "7a865a2588e3488c9dd1b815909c3f3c"
format_stamp: "Formatted at 2026-08-07 19:31:56 on dist-test-slave-8w9v"
server_key: "dff4256d780922051716a4e26a353b51"
server_key_iv: "c47741170bf34e5c8cabbb09910eeffe"
server_key_version: "encryptionkey@0"
I20260807 19:31:56.280586 18265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "7a865a2588e3488c9dd1b815909c3f3c"
format_stamp: "Formatted at 2026-08-07 19:31:56 on dist-test-slave-8w9v"
server_key: "dff4256d780922051716a4e26a353b51"
server_key_iv: "c47741170bf34e5c8cabbb09910eeffe"
server_key_version: "encryptionkey@0"
I20260807 19:31:56.285791 18265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:31:56.289354 18283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:56.290848 18265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:31:56.291021 18265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "7a865a2588e3488c9dd1b815909c3f3c"
format_stamp: "Formatted at 2026-08-07 19:31:56 on dist-test-slave-8w9v"
server_key: "dff4256d780922051716a4e26a353b51"
server_key_iv: "c47741170bf34e5c8cabbb09910eeffe"
server_key_version: "encryptionkey@0"
I20260807 19:31:56.291244 18265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:56.297991 18116 heartbeater.cc:503] Master 127.8.216.190:33351 was elected leader, sending a full tablet report...
I20260807 19:31:56.312831 18265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:56.317238 18265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:56.317567 18265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:56.318594 18265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:56.320209 18265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:56.320323 18265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:31:56.320444 18265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:56.320492 18265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:56.357371 18265 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:34011
I20260807 19:31:56.357424 18401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:34011 every 8 connection(s)
I20260807 19:31:56.358728 18265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:56.364338 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18265
I20260807 19:31:56.364559 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:31:56.364990 9058 external_mini_cluster.cc:1428] Setting key f5de0f475223082f3d3c8ec8401f117b
Aug 07 19:31:56 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131116, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:31:56.380481 18404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:56.361477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:47637 (local address 127.8.216.190:33351)
0807 19:31:56.361975 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:56.361980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:56.362764 (+ 784us) server_negotiation.cc:407] Connection header received
0807 19:31:56.364770 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:56.364776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:56.364883 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:56.365021 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:56.366981 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.368084 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.369685 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.370108 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.370206 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:56.374126 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:56.374131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:56.374134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:56.374168 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:56.376984 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.377769 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.377773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.377776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.377898 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.378384 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.378387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.378389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.378753 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:56.378925 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:56.379803 (+ 878us) server_negotiation.cc:299] Negotiation successful
0807 19:31:56.380061 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":273,"threads_started":1}
I20260807 19:31:56.381541 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:56.361855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:47637)
0807 19:31:56.362591 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:56.362630 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:56.364412 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:56.365275 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:56.365283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:56.365980 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:56.366736 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:56.366758 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.368278 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.368282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:56.369547 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:56.369560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.370302 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.370307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:56.370523 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:56.371263 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:56.371290 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:56.373911 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:31:56.377133 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:56.377141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:56.377156 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:56.377659 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:56.378048 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:31:56.378051 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:31:56.378054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:31:56.378273 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:31:56.378931 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:31:56.378949 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:31:56.379361 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 19:31:56.379669 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 19:31:56.380286 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":275,"threads_started":1}
I20260807 19:31:56.383281 18402 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33351
I20260807 19:31:56.383611 18402 heartbeater.cc:464] Registering TS with master...
I20260807 19:31:56.384473 18402 heartbeater.cc:511] Master 127.8.216.190:33351 requested a full tablet report, sending...
I20260807 19:31:56.386312 17914 ts_manager.cc:194] Registered new tserver with Master: 7a865a2588e3488c9dd1b815909c3f3c (127.8.216.131:34011)
I20260807 19:31:56.387346 17914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:47637
I20260807 19:31:56.393972 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
Entry for principal oryx/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab.
WARNING: no policy specified for oryx/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
Entry for principal oryx/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab.
WARNING: no policy specified for oryx/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
Entry for principal oryx/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab.
I20260807 19:31:56.567420 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17978
Aug 07 19:31:56 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:31:56.584867 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:56.568673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.8.216.129:38049)
0807 19:31:56.569391 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:56.569399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:56.569427 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:31:56.569490 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:56.569497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:56.569722 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:56.569860 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:56.571249 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.572437 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.573693 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.574281 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.574402 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:56.578766 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:56.578781 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:56.578796 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:56.578841 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:56.582054 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.582789 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.582797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.582799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.582928 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.583255 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.583258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.583260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.583624 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:56.583857 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:56.584218 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 19:31:56.584450 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":371,"threads_started":1}
I20260807 19:31:56.586252 18111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:56.677348 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17978
I20260807 19:31:56.698738 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18122
Aug 07 19:31:56 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:31:56.716223 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:56.699803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.8.216.130:40165)
0807 19:31:56.700489 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:56.700497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:56.700524 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:31:56.700585 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:56.700591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:56.700802 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:56.700934 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:56.702319 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.703384 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.704937 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.705522 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.705628 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:56.710143 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:56.710158 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:56.710172 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:56.710210 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:56.713314 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.714161 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.714169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.714171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.714307 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.714659 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.714662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.714664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.715040 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:56.715289 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:56.715594 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 19:31:56.715818 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":381,"threads_started":1}
I20260807 19:31:56.717665 18254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:56.805315 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18122
I20260807 19:31:56.824991 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18265
Aug 07 19:31:56 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:31:56.841487 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:56.826034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.8.216.131:34011)
0807 19:31:56.826651 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:56.826658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:56.826689 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:31:56.826755 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:56.826782 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:56.827060 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:56.827193 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:56.828653 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.829698 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.830928 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:56.831515 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:56.831614 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:56.836195 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:56.836208 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:56.836223 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:56.836263 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:56.838891 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.839542 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.839550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.839552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.839679 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:56.840000 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:56.840003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:56.840005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:56.840446 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:56.840679 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:56.840921 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 19:31:56.841173 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":295,"threads_started":1}
I20260807 19:31:56.842819 18397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:56.930241 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18265
I20260807 19:31:56.950147 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 17879
I20260807 19:31:56.951423 17945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:31:57.059612 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 17879
I20260807 19:31:57.085091 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33351
--webserver_interface=127.8.216.190
--webserver_port=45259
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:31:57.427610 18422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:57.427996 18422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:57.428231 18422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:57.435521 18422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:31:57.435676 18422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:57.435788 18422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:57.435907 18422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:31:57.436015 18422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:31:57.446120 18422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=45259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:57.447516 18422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:57.448952 18422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:57.461532 18428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:57.461661 18430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:57.461942 18427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:57.462709 18422 server_base.cc:1034] running on GCE node
I20260807 19:31:57.463514 18422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:57.465091 18422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:57.466297 18422 hybrid_clock.cc:648] HybridClock initialized: now 1786131117466250 us; error 45 us; skew 500 ppm
I20260807 19:31:57.466674 18422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:57 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131117, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:57.469874 18422 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:31:57.471429 18422 webserver.cc:460] Webserver started at http://127.8.216.190:45259/ using document root <none> and password file <none>
I20260807 19:31:57.472103 18422 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:57.472280 18422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:57.477859 18422 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 19:31:57.480793 18437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:57.482400 18422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:31:57.482609 18422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "52bd920b59494593b689f58a6b0ccd7d"
format_stamp: "Formatted at 2026-08-07 19:31:54 on dist-test-slave-8w9v"
server_key: "3bedb1faf47ebafb57d09389cc1474ec"
server_key_iv: "c4edbc8958175b1e40f301e1786e6bcd"
server_key_version: "encryptionkey@0"
I20260807 19:31:57.483662 18422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:57.508298 18422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:57.512974 18422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:57.513271 18422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:57.533496 18422 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33351
I20260807 19:31:57.533529 18493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33351 every 8 connection(s)
I20260807 19:31:57.535105 18422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:31:57.542264 18494 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:31:57.543035 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18422
I20260807 19:31:57.543855 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:38049
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:33351
--webserver_port=33941
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:31:57.545823 18494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Bootstrap starting.
I20260807 19:31:57.569666 18494 log.cc:824] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:31:57.582211 18494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:31:57.582926 18494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Bootstrap complete.
I20260807 19:31:57.597313 18494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:57.598613 18494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52bd920b59494593b689f58a6b0ccd7d, State: Initialized, Role: FOLLOWER
I20260807 19:31:57.599406 18494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:57.599702 18494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:31:57.599825 18494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:31:57.600020 18494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:31:57.605836 18494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:57.606555 18494 leader_election.cc:302] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52bd920b59494593b689f58a6b0ccd7d; no voters:
I20260807 19:31:57.607259 18494 leader_election.cc:288] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:31:57.607455 18498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:31:57.608788 18498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [term 2 LEADER]: Becoming Leader. State: Replica: 52bd920b59494593b689f58a6b0ccd7d, State: Running, Role: LEADER
I20260807 19:31:57.609356 18498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } }
I20260807 19:31:57.609702 18494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:31:57.612205 18499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "52bd920b59494593b689f58a6b0ccd7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } } }
I20260807 19:31:57.612175 18500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52bd920b59494593b689f58a6b0ccd7d. Latest consensus state: current_term: 2 leader_uuid: "52bd920b59494593b689f58a6b0ccd7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52bd920b59494593b689f58a6b0ccd7d" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33351 } } }
I20260807 19:31:57.613123 18499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:57.613147 18500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d [sys.catalog]: This master's current role is: LEADER
I20260807 19:31:57.623787 18505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:31:57.630064 18505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:31:57.631551 18505 catalog_manager.cc:1259] Loaded cluster ID: 54e35538e0224e69993113876f519732
I20260807 19:31:57.631668 18505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:31:57.635592 18505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:31:57.637122 18505 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52bd920b59494593b689f58a6b0ccd7d: Loaded TSK: 0
I20260807 19:31:57.638604 18505 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:31:57.928689 18496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:57.929019 18496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:57.929234 18496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:57.936836 18496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:57.936997 18496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:57.937172 18496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:31:57.947516 18496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.129.keytab
--principal=oryx/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:38049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=33941
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:57.948984 18496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:57.950363 18496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:57.965339 18522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:57.965687 18523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:57.966785 18496 server_base.cc:1034] running on GCE node
W20260807 19:31:57.966915 18525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:57.967738 18496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:57.969062 18496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:57.970314 18496 hybrid_clock.cc:648] HybridClock initialized: now 1786131117970257 us; error 50 us; skew 500 ppm
I20260807 19:31:57.970706 18496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:57 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131117, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:57.978189 18496 init.cc:377] Logged in from keytab as oryx/127.8.216.129@KRBTEST.COM (short username oryx)
I20260807 19:31:57.979894 18496 webserver.cc:460] Webserver started at http://127.8.216.129:33941/ using document root <none> and password file <none>
I20260807 19:31:57.980865 18496 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:57.980993 18496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:57.987113 18496 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 19:31:57.990327 18533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:57.991881 18496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:31:57.992031 18496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "7e22874e186744b1bec933f83341eb15"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "37b0959e16049a28a1e33cc5ee35085a"
server_key_iv: "7c3ac5b1bf86edcff1a5fe10a476ce80"
server_key_version: "encryptionkey@0"
I20260807 19:31:57.993088 18496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:58.020620 18496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:58.024803 18496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:58.025040 18496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:58.026024 18496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:58.027449 18496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:58.027552 18496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:58.027635 18496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:58.027693 18496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:58.062633 18496 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38049
I20260807 19:31:58.062726 18652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38049 every 8 connection(s)
I20260807 19:31:58.064118 18496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:31:58.068173 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18496
I20260807 19:31:58.069021 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:40165
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:33351
--webserver_port=44991
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:31:58 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:58 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:31:58.419308 18658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:58.419620 18658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:58.419771 18658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:31:58.427379 18658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:58.427493 18658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:58.427713 18658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:31:58.437924 18658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.130.keytab
--principal=oryx/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:40165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=44991
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:58.439316 18658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:58.440860 18658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:58.455703 18664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:58.456297 18665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:58.455840 18667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:58.456816 18658 server_base.cc:1034] running on GCE node
I20260807 19:31:58.457775 18658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:58.458827 18658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:58.460265 18658 hybrid_clock.cc:648] HybridClock initialized: now 1786131118460200 us; error 96 us; skew 500 ppm
I20260807 19:31:58.460599 18658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:58 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131118, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:58.463951 18658 init.cc:377] Logged in from keytab as oryx/127.8.216.130@KRBTEST.COM (short username oryx)
I20260807 19:31:58.465627 18658 webserver.cc:460] Webserver started at http://127.8.216.130:44991/ using document root <none> and password file <none>
I20260807 19:31:58.466431 18658 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:58.466579 18658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:58.472745 18658 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:31:58.475992 18674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:58.477739 18658 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:31:58.477937 18658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "3a02b88628584681a5fe01ae157cd3e0"
format_stamp: "Formatted at 2026-08-07 19:31:55 on dist-test-slave-8w9v"
server_key: "a56c3fa10a513b938987ecd75fb3564b"
server_key_iv: "1212e8d2d4605580af6ed072534a5894"
server_key_version: "encryptionkey@0"
I20260807 19:31:58.478951 18658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:58.517369 18658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:58.522209 18658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:58.522522 18658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:58.523679 18658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:58.525336 18658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:58.525489 18658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:58.525624 18658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:58.525714 18658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:58.576218 18658 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:40165
I20260807 19:31:58.576233 18795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:40165 every 8 connection(s)
I20260807 19:31:58.579084 18658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:31:58.588449 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18658
I20260807 19:31:58.589205 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:34011
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:33351
--webserver_port=46373
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:31:58 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:58 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:31:59.288051 18655 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.210s user 0.001s sys 0.014s
I20260807 19:31:59.290277 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:58.067222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:37357)
0807 19:31:58.068089 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:58.068167 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:58.070250 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:58.071708 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:58.071721 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:58.072432 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:58.072969 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:58.072987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.074672 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.074676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:58.075906 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:58.075919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.077056 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.077064 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:58.077400 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:58.078204 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:58.078239 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.289051 (+1210812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":368,"threads_started":1}
W20260807 19:31:59.290271 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:58.067212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:37357 (local address 127.8.216.190:33351)
0807 19:31:58.069026 (+ 1814us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:58.069043 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:58.069094 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:31:58.070845 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:58.070859 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:58.071183 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:58.071578 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:58.073206 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.074486 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.076078 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.076792 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.076996 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.289467 (+1212471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:37357: BlockingRecv error: recv got EOF from 127.8.216.129:37357 (error 108)
Metrics: {"server-negotiator.queue_time_us":1557,"thread_start_us":1308,"threads_started":1}
W20260807 19:31:59.291519 18653 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:33351 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.216.190:33351: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
W20260807 19:31:59.295908 18800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:31:59.296528 18800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:31:59.296780 18800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:31:59.302001 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.292284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:36205)
0807 19:31:59.292751 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:59.292772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:59.292911 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:59.293274 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:59.293279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:59.293676 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:59.293843 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.293851 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.294955 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.294963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.295930 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.295944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.296985 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.296991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.297159 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.297849 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:59.297866 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.301678 (+ 3812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325}
W20260807 19:31:59.302186 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:59.292666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36205 (local address 127.8.216.190:33351)
0807 19:31:59.292833 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.292839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.292859 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:31:59.293064 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.293069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.293133 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.293244 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.294000 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.294808 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.296096 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.296761 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.296896 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.301916 (+ 5020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:36205: BlockingRecv error: recv got EOF from 127.8.216.129:36205 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:31:59.311606 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.302958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:44829)
0807 19:31:59.303116 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:59.303137 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:59.303241 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:59.303665 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:59.303671 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:59.303994 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:59.304218 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.304228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.305247 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.305252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.306186 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.306197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.306899 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.306905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.307089 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.307693 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:59.307710 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.311299 (+ 3589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:31:59.311664 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:59.303210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44829 (local address 127.8.216.190:33351)
0807 19:31:59.303375 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.303380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.303402 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:31:59.303447 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.303451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.303519 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.303626 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.304356 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.305100 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.306327 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.306744 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.306868 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.311435 (+ 4567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:44829: BlockingRecv error: recv got EOF from 127.8.216.129:44829 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 19:31:59.312544 18653 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:31:59.316365 18800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:31:59.316578 18800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:31:59.316887 18800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:31:59.342597 18800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/krb5kdc/oryx_127.8.216.131.keytab
--principal=oryx/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:34011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=46373
--tserver_master_addrs=127.8.216.190:33351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:31:59.345218 18800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:31:59.347729 18800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:31:59.367916 18807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:31:59.368319 18809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:59.368817 18800 server_base.cc:1034] running on GCE node
W20260807 19:31:59.372593 18806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:31:59.373481 18800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:31:59.374601 18800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:31:59.376108 18800 hybrid_clock.cc:648] HybridClock initialized: now 1786131119376005 us; error 95 us; skew 500 ppm
I20260807 19:31:59.376502 18800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131119, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:31:59.380402 18800 init.cc:377] Logged in from keytab as oryx/127.8.216.131@KRBTEST.COM (short username oryx)
I20260807 19:31:59.382296 18800 webserver.cc:460] Webserver started at http://127.8.216.131:46373/ using document root <none> and password file <none>
I20260807 19:31:59.383242 18800 fs_manager.cc:359] Metadata directory not provided
I20260807 19:31:59.383411 18800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:31:59.389412 18800 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:31:59.392760 18816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:59.394555 18800 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:31:59.394753 18800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "7a865a2588e3488c9dd1b815909c3f3c"
format_stamp: "Formatted at 2026-08-07 19:31:56 on dist-test-slave-8w9v"
server_key: "dff4256d780922051716a4e26a353b51"
server_key_iv: "c47741170bf34e5c8cabbb09910eeffe"
server_key_version: "encryptionkey@0"
I20260807 19:31:59.395784 18800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:31:59.414822 18800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:31:59.419395 18800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:31:59.419691 18800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:31:59.420876 18800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:31:59.422355 18800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:31:59.422487 18800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:59.422617 18800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:31:59.422704 18800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:31:59.471320 18800 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:34011
I20260807 19:31:59.471369 18935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:34011 every 8 connection(s)
I20260807 19:31:59.472954 18800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:31:59.475034 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18800
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:31:59.517495 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.476377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.8.216.190:33351)
0807 19:31:59.492499 (+ 16122us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.492506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.492527 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:31:59.492593 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.492599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.492682 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.492800 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.495896 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.497219 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.500066 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.500737 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.500889 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.504757 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:31:59.504766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:31:59.504785 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:31:59.504820 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:31:59.511959 (+ 7139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:59.513350 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:59.513363 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:59.513375 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:59.513629 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:31:59.514256 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:31:59.514263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:31:59.514267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:31:59.515048 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:31:59.515393 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 19:31:59.516105 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 19:31:59.516911 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15989,"thread_start_us":16294,"threads_started":1}
W20260807 19:31:59.871142 18799 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.275s user 0.009s sys 0.019s
W20260807 19:31:59.872908 18798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:58.582529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:54849 (local address 127.8.216.190:33351)
0807 19:31:58.583144 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:58.583150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:58.584193 (+ 1043us) server_negotiation.cc:407] Connection header received
0807 19:31:58.586307 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:58.586319 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:58.586474 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:58.586670 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:58.588859 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.590699 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.593394 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.594143 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.594262 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.872616 (+1278354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:54849: BlockingRecv error: recv got EOF from 127.8.216.130:54849 (error 108)
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":301,"threads_started":1}
I20260807 19:31:59.874305 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:58.582996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:54849)
0807 19:31:58.583989 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:58.584047 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:58.585980 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:58.586887 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:58.586895 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:58.587614 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:58.588526 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:58.588559 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.590942 (+ 2383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.590950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:58.593194 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:58.593212 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:58.594694 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:58.594705 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:58.595048 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:58.595908 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:58.595951 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.872508 (+1276557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":413,"threads_started":1}
W20260807 19:31:59.875360 18796 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:33351 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.216.190:33351: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:31:59.889554 18798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:59.876354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:48465 (local address 127.8.216.190:33351)
0807 19:31:59.876519 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.876523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.876540 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:31:59.876702 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.876706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.876765 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.876850 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.878227 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.878994 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.881732 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.882466 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.882641 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.889333 (+ 6692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:48465: BlockingRecv error: recv got EOF from 127.8.216.130:48465 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:31:59.889600 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.876051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:48465)
0807 19:31:59.876361 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:59.876390 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:59.876559 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:59.876946 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:59.876954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:59.877682 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:59.878023 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.878040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.879156 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.879164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.881505 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.881528 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.882783 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.882793 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.883090 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.884018 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:59.884048 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.889085 (+ 5037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:31:59 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:31:59.903829 18799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.891096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:52501)
0807 19:31:59.891466 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:59.891492 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:59.891643 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:59.892263 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:59.892272 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:59.892946 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:59.893268 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.893287 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.894394 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.894403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.896974 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.896999 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.897798 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.897805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.898139 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.898929 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:59.898963 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:31:59.903618 (+ 4655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215}
W20260807 19:31:59.904268 18796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:31:59.904552 18798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:59.891314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:52501 (local address 127.8.216.190:33351)
0807 19:31:59.891554 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.891562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.891586 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:31:59.891808 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.891834 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.891950 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.892072 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.893485 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.894218 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.897153 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.897617 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.897713 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.904081 (+ 6368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:52501: BlockingRecv error: recv got EOF from 127.8.216.130:52501 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:00.325757 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:00.313202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:58461)
0807 19:32:00.313800 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:00.313817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:00.313936 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:00.314420 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:00.314423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:00.314888 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:00.315040 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.315049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.317054 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.317058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.318128 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.318141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.319545 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.319550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.319674 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.320369 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:00.320389 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:00.325222 (+ 4833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":347,"threads_started":1}
W20260807 19:32:00.325773 18798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:00.313341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:58461 (local address 127.8.216.190:33351)
0807 19:32:00.313607 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:00.313616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:00.313908 (+ 292us) server_negotiation.cc:407] Connection header received
0807 19:32:00.314088 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:00.314108 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:00.314247 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:00.314402 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:00.315203 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.316891 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.318301 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.319334 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.319517 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.325475 (+ 5958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:58461: BlockingRecv error: recv got EOF from 127.8.216.129:58461 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:32:00.617221 18940 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.114s user 0.005s sys 0.013s
W20260807 19:32:00.618399 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:31:59.475919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54055 (local address 127.8.216.190:33351)
0807 19:31:59.476203 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 19:31:59.476209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:31:59.491335 (+ 15126us) server_negotiation.cc:407] Connection header received
0807 19:31:59.494738 (+ 3403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:31:59.494744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:31:59.494829 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:31:59.494927 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:31:59.496999 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.497804 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.499488 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.500172 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.500284 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.618191 (+1117907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:54055: BlockingRecv error: recv got EOF from 127.8.216.131:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 19:32:00.618942 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:31:59.476711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:54055)
0807 19:31:59.491049 (+ 14338us) negotiation.cc:107] Waiting for socket to connect
0807 19:31:59.491133 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 19:31:59.494427 (+ 3294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:31:59.495220 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:31:59.495229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:31:59.495837 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:31:59.496770 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.496798 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.498043 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.498048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.499300 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:31:59.499312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:31:59.500540 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:31:59.500551 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:31:59.501016 (+ 465us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:31:59.502662 (+ 1646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:31:59.502719 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:00.618003 (+1115284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13796,"thread_start_us":14518,"threads_started":1}
W20260807 19:32:00.619863 18936 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:33351 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.216.190:33351: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:00.629498 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:00.620555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:58043)
0807 19:32:00.620748 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:00.620782 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:00.620878 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:00.621187 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:00.621190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:00.621538 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:00.621712 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.621720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.622755 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.622759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.623714 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.623725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.624524 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.624529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.624650 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.625301 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:00.625323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:00.629188 (+ 3865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 19:32:00.629705 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:00.620701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:58043 (local address 127.8.216.190:33351)
0807 19:32:00.620923 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:00.620928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:00.620944 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:32:00.620995 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:00.620999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:00.621059 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:00.621149 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:00.621848 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.622634 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.623843 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.624360 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.624463 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.629427 (+ 4964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:58043: BlockingRecv error: recv got EOF from 127.8.216.131:58043 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:00.638970 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:00.630416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:59605)
0807 19:32:00.630622 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:00.630639 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:00.630738 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:00.631041 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:00.631045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:00.631356 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:00.631512 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.631520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.632656 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.632660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.633647 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.633660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.634306 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.634310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.634436 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.635002 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:00.635020 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:00.638697 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:32:00.639215 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:00.630491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:59605 (local address 127.8.216.190:33351)
0807 19:32:00.630698 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:00.630702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:00.630718 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:32:00.630847 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:00.630851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:00.630912 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:00.631000 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:00.631666 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.632522 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.633772 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.634178 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.634269 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.638957 (+ 4688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:59605: BlockingRecv error: recv got EOF from 127.8.216.131:59605 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 19:32:00.639346 18936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:00 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:00.914645 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:00.904936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:38711)
0807 19:32:00.905561 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:00.905583 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:00.905721 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:00.906111 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:00.906116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:00.906622 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:00.906801 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.906811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.907851 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.907854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.908925 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:00.908981 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.909851 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.909857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:00.909981 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.910704 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:00.910722 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:00.914256 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":382,"threads_started":1}
W20260807 19:32:00.914744 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:00.905044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:38711 (local address 127.8.216.190:33351)
0807 19:32:00.905234 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:00.905238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:00.905669 (+ 431us) server_negotiation.cc:407] Connection header received
0807 19:32:00.905851 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:00.905855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:00.905922 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:00.906011 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:00.906968 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.907710 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.909107 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:00.909590 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:00.909715 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:00.914492 (+ 4777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:38711: BlockingRecv error: recv got EOF from 127.8.216.130:38711 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:01.337770 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:01.327001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:50051)
0807 19:32:01.327571 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:01.327588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:01.327721 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:01.328041 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:01.328045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:01.328511 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:01.328709 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.328718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.329878 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.329881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.330774 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.330785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.331924 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.331932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.332217 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.333035 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:01.333057 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:01.337319 (+ 4262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":339,"threads_started":1}
W20260807 19:32:01.337894 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:01.327103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:50051 (local address 127.8.216.190:33351)
0807 19:32:01.327299 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:01.327304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:01.327674 (+ 370us) server_negotiation.cc:407] Connection header received
0807 19:32:01.327837 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:01.327841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:01.327907 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:01.327996 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:01.328882 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.329746 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.330977 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.331721 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.331880 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.337554 (+ 5674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:50051: BlockingRecv error: recv got EOF from 127.8.216.129:50051 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:01.649086 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:01.639971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:42799)
0807 19:32:01.640593 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:01.640611 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:01.640730 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:01.641020 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:01.641023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:01.641483 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:01.641724 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.641734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.642736 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.642740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.643686 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.643696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.644394 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.644398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.644531 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.645106 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:01.645123 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:01.648698 (+ 3575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":332,"threads_started":1}
W20260807 19:32:01.649207 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:01.640031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:42799 (local address 127.8.216.190:33351)
0807 19:32:01.640308 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:01.640318 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:01.640673 (+ 355us) server_negotiation.cc:407] Connection header received
0807 19:32:01.640829 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:01.640833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:01.640898 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:01.640971 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:01.641854 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.642612 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.643815 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.644224 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.644323 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.648928 (+ 4605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:42799: BlockingRecv error: recv got EOF from 127.8.216.131:42799 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:01 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:01.927937 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:01.915650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:41811)
0807 19:32:01.916286 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:01.916335 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:01.916564 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:01.917075 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:01.917081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:01.917691 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:01.918019 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.918032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.919746 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.919753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.921553 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:01.921570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.922667 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.922673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:01.922903 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.923901 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:01.923926 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:01.927682 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":369,"threads_started":1}
W20260807 19:32:01.928318 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:01.915732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:41811 (local address 127.8.216.190:33351)
0807 19:32:01.916023 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:01.916029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:01.916424 (+ 395us) server_negotiation.cc:407] Connection header received
0807 19:32:01.916814 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:01.916821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:01.916906 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:01.917005 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:01.918207 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.919533 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.921717 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:01.922489 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:01.922672 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:01.927899 (+ 5227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:41811: BlockingRecv error: recv got EOF from 127.8.216.130:41811 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:02.348945 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:02.338887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:41443)
0807 19:32:02.339455 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:02.339475 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:02.339609 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:02.339954 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:02.339957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:02.340502 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:02.340689 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.340700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.341891 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.341895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.342875 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.342887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.343946 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.343952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.344250 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.344990 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:02.345010 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:02.348579 (+ 3569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":309,"threads_started":1}
W20260807 19:32:02.349037 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:02.339103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:41443 (local address 127.8.216.190:33351)
0807 19:32:02.339357 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:02.339363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:02.339558 (+ 195us) server_negotiation.cc:407] Connection header received
0807 19:32:02.339719 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:02.339723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:02.339788 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:02.339913 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:02.340847 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.341729 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.343042 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.343795 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.343934 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.348800 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:41443: BlockingRecv error: recv got EOF from 127.8.216.129:41443 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:02.660037 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:02.650157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:42057)
0807 19:32:02.650697 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:02.650714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:02.650831 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:02.651162 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:02.651165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:02.651584 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:02.651828 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.651842 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.653053 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.653060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.654085 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.654100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.654925 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.654931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.655157 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.655887 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:02.655908 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:02.659612 (+ 3704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":317,"threads_started":1}
W20260807 19:32:02.660329 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:02.650280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:42057 (local address 127.8.216.190:33351)
0807 19:32:02.650576 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:02.650583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:02.650809 (+ 226us) server_negotiation.cc:407] Connection header received
0807 19:32:02.650951 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:02.650957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:02.651033 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:02.651122 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:02.652002 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.652855 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.654264 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.654742 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.654899 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.659862 (+ 4963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:42057: BlockingRecv error: recv got EOF from 127.8.216.131:42057 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:02 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:02.938568 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:02.928954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:44941)
0807 19:32:02.929482 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:02.929502 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:02.929670 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:02.930034 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:02.930039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:02.930427 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:02.930614 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.930623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.931962 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.931967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.933077 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:02.933090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.933819 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.933825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:02.934015 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.934743 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:02.934769 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:02.938290 (+ 3521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":280,"threads_started":1}
W20260807 19:32:02.938918 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:02.929061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:44941 (local address 127.8.216.190:33351)
0807 19:32:02.929303 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:02.929310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:02.929597 (+ 287us) server_negotiation.cc:407] Connection header received
0807 19:32:02.929802 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:02.929810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:02.929880 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:02.929997 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:02.930782 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.931828 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.933228 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:02.933664 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:02.933793 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:02.938516 (+ 4723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:44941: BlockingRecv error: recv got EOF from 127.8.216.130:44941 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:03.361874 18950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:03.350096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:38965)
0807 19:32:03.350663 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:03.350680 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:03.350796 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:03.351106 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:03.351109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:03.351525 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:03.351698 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.351707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.352713 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.352717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.353728 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.353740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.354389 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.354394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.354514 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.358000 (+ 3486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:03.358020 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:03.361605 (+ 3585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":319,"threads_started":1}
W20260807 19:32:03.362028 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:03.350213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:38965 (local address 127.8.216.190:33351)
0807 19:32:03.350385 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:03.350390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:03.350758 (+ 368us) server_negotiation.cc:407] Connection header received
0807 19:32:03.350899 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:03.350902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:03.350973 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:03.351054 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:03.351826 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.352600 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.353852 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.354258 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.354348 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.361803 (+ 7455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:38965: BlockingRecv error: recv got EOF from 127.8.216.129:38965 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:03.670572 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:03.661299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:47681)
0807 19:32:03.661866 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:03.661883 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:03.662059 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:03.662413 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:03.662418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:03.662839 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:03.663117 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.663129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.664364 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.664371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.665451 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.665476 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.666139 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.666143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.666262 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.666903 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:03.666921 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:03.670334 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":319,"threads_started":1}
W20260807 19:32:03.670784 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:03.661467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:47681 (local address 127.8.216.190:33351)
0807 19:32:03.661719 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:03.661725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:03.661979 (+ 254us) server_negotiation.cc:407] Connection header received
0807 19:32:03.662190 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:03.662200 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:03.662264 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:03.662376 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:03.663277 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.664222 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.665591 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.666017 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.666111 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.670526 (+ 4415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:47681: BlockingRecv error: recv got EOF from 127.8.216.131:47681 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:03 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:03.949116 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:03.939666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:48411)
0807 19:32:03.940278 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:03.940300 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:03.940427 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:03.940750 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:03.940753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:03.941250 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:03.941431 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.941442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.942462 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.942465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.943445 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:03.943456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.944216 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.944222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:03.944426 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.945054 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:03.945078 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:03.948850 (+ 3772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":344,"threads_started":1}
W20260807 19:32:03.949461 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:03.939786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:48411 (local address 127.8.216.190:33351)
0807 19:32:03.940086 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:03.940093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:03.940413 (+ 320us) server_negotiation.cc:407] Connection header received
0807 19:32:03.940537 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:03.940541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:03.940606 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:03.940699 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:03.941602 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.942340 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.943570 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:03.944005 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:03.944173 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:03.949110 (+ 4937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:48411: BlockingRecv error: recv got EOF from 127.8.216.130:48411 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:04.372293 18953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:04.362979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:44081)
0807 19:32:04.363626 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:04.363648 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:04.363818 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:04.364244 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:04.364249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:04.364843 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:04.365072 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.365086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.366269 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.366273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.367249 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.367261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.367934 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.367938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.368057 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.368672 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:04.368689 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:04.372029 (+ 3340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
W20260807 19:32:04.372507 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:04.363196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44081 (local address 127.8.216.190:33351)
0807 19:32:04.363425 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:04.363432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:04.363780 (+ 348us) server_negotiation.cc:407] Connection header received
0807 19:32:04.363971 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:04.363980 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:04.364075 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:04.364190 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:04.365254 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.366148 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.367376 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.367809 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.367901 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.372279 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:44081: BlockingRecv error: recv got EOF from 127.8.216.129:44081 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:04.680871 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:04.671733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:44937)
0807 19:32:04.672327 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:04.672344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:04.672492 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:04.672887 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:04.672890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:04.673273 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:04.673443 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.673452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.674402 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.674406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.675342 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.675352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.675977 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.675981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.676121 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.676731 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:04.676748 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:04.680613 (+ 3865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":325,"threads_started":1}
W20260807 19:32:04.681048 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:04.671827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:44937 (local address 127.8.216.190:33351)
0807 19:32:04.672042 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:04.672047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:04.672430 (+ 383us) server_negotiation.cc:407] Connection header received
0807 19:32:04.672619 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:04.672627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:04.672744 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:04.672828 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:04.673603 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.674291 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.675464 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.675848 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.675934 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.680809 (+ 4875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:44937: BlockingRecv error: recv got EOF from 127.8.216.131:44937 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:04 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:04.959281 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:04.950161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:50733)
0807 19:32:04.950714 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:04.950738 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:04.950901 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:04.951256 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:04.951261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:04.951749 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:04.951909 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.951917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.953037 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.953043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.953982 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:04.953993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.954722 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.954728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:04.954915 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.955526 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:04.955543 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:04.958955 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":306,"threads_started":1}
W20260807 19:32:04.959477 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:04.950404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:50733 (local address 127.8.216.190:33351)
0807 19:32:04.950611 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:04.950618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:04.950826 (+ 208us) server_negotiation.cc:407] Connection header received
0807 19:32:04.951032 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:04.951038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:04.951111 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:04.951217 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:04.952072 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.952880 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.954131 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:04.954563 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:04.954695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:04.959201 (+ 4506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:50733: BlockingRecv error: recv got EOF from 127.8.216.130:50733 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:05.382969 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:05.373507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:47581)
0807 19:32:05.373990 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:05.374007 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:05.374138 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:05.374485 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:05.374488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:05.374926 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:05.375104 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.375112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.376201 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.376208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.377258 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.377271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.378227 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.378234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.378411 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.379225 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:05.379254 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:05.382687 (+ 3433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":274,"threads_started":1}
W20260807 19:32:05.383175 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:05.373594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:47581 (local address 127.8.216.190:33351)
0807 19:32:05.373843 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:05.373849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:05.374107 (+ 258us) server_negotiation.cc:407] Connection header received
0807 19:32:05.374254 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:05.374259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:05.374347 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:05.374436 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:05.375235 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.375987 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.377418 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.378033 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.378153 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.382912 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:47581: BlockingRecv error: recv got EOF from 127.8.216.129:47581 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:05.693670 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:05.681953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:54681)
0807 19:32:05.682503 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:05.682524 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:05.682676 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:05.683177 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:05.683182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:05.683742 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:05.683940 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.683951 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.685692 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.685697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.687004 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.687020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.687939 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.687945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.688174 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.688872 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:05.688902 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:05.693334 (+ 4432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":391,"threads_started":1}
W20260807 19:32:05.693809 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:05.682030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54681 (local address 127.8.216.190:33351)
0807 19:32:05.682348 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:05.682355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:05.682599 (+ 244us) server_negotiation.cc:407] Connection header received
0807 19:32:05.682793 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:05.682799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:05.682886 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:05.682977 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:05.684109 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.685491 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.687171 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.687784 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.687915 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.693546 (+ 5631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:54681: BlockingRecv error: recv got EOF from 127.8.216.131:54681 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:05 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:05.968973 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:05.960303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:51343)
0807 19:32:05.960826 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:05.960849 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:05.960968 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:05.961267 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:05.961270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:05.961726 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:05.961895 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.961904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.962929 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.962933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.963872 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:05.963882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.964574 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.964578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:05.964714 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.965382 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:05.965400 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:05.968744 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":295,"threads_started":1}
W20260807 19:32:05.969203 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:05.960400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:51343 (local address 127.8.216.190:33351)
0807 19:32:05.960605 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:05.960611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:05.960923 (+ 312us) server_negotiation.cc:407] Connection header received
0807 19:32:05.961066 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:05.961070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:05.961134 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:05.961217 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:05.962026 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.962792 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.963997 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:05.964413 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:05.964519 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:05.968980 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:51343: BlockingRecv error: recv got EOF from 127.8.216.130:51343 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:06.393590 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:06.384024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:51801)
0807 19:32:06.384692 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:06.384710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:06.384839 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:06.385155 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:06.385158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:06.385586 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:06.385761 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.385770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.386967 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.386971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.388311 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.388323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.389005 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.389009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.389119 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.389747 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:06.389764 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:06.393309 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":272,"threads_started":1}
W20260807 19:32:06.393783 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:06.384105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:51801 (local address 127.8.216.190:33351)
0807 19:32:06.384392 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:06.384403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:06.384798 (+ 395us) server_negotiation.cc:407] Connection header received
0807 19:32:06.384952 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:06.384956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:06.385024 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:06.385111 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:06.385891 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.386809 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.388457 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.388872 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.388974 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.393552 (+ 4578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:51801: BlockingRecv error: recv got EOF from 127.8.216.129:51801 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:06.704391 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:06.694939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:54153)
0807 19:32:06.695458 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:06.695474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:06.695639 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:06.696016 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:06.696020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:06.696493 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:06.696668 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.696677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.697799 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.697805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.698885 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.698898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.699636 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.699641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.699824 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.700532 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:06.700552 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:06.704003 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":271,"threads_started":1}
W20260807 19:32:06.704567 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:06.695069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54153 (local address 127.8.216.190:33351)
0807 19:32:06.695296 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:06.695304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:06.695563 (+ 259us) server_negotiation.cc:407] Connection header received
0807 19:32:06.695767 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:06.695773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:06.695852 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:06.695957 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:06.696832 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.697650 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.699034 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.699470 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.699609 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.704284 (+ 4675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:54153: BlockingRecv error: recv got EOF from 127.8.216.131:54153 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:06 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:06.979131 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:06.969897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:55541)
0807 19:32:06.970517 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:06.970540 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:06.970655 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:06.970951 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:06.970954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:06.971370 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:06.971542 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.971550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.972584 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.972588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.973528 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:06.973540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.974227 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.974231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:06.974344 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.974984 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:06.975002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:06.978888 (+ 3886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":292,"threads_started":1}
W20260807 19:32:06.979318 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:06.970004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:55541 (local address 127.8.216.190:33351)
0807 19:32:06.970223 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:06.970228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:06.970614 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:32:06.970754 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:06.970758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:06.970820 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:06.970904 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:06.971664 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.972461 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.973654 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:06.974097 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:06.974191 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:06.979098 (+ 4907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:55541: BlockingRecv error: recv got EOF from 127.8.216.130:55541 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:07.403847 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:07.394702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:36303)
0807 19:32:07.395294 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:07.395315 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:07.395469 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:07.395838 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:07.395842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:07.396323 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:07.396499 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.396508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.397695 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.397698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.398707 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.398718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.399410 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.399414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.399525 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.400235 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:07.400253 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:07.403603 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":322,"threads_started":1}
W20260807 19:32:07.404115 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:07.394776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36303 (local address 127.8.216.190:33351)
0807 19:32:07.395008 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:07.395016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:07.395405 (+ 389us) server_negotiation.cc:407] Connection header received
0807 19:32:07.395615 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:07.395620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:07.395700 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:07.395790 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:07.396626 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.397562 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.398829 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.399237 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.399333 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.403861 (+ 4528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:36303: BlockingRecv error: recv got EOF from 127.8.216.129:36303 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:07.715382 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:07.705456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:38499)
0807 19:32:07.706032 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:07.706048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:07.706162 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:07.706445 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:07.706448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:07.706859 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:07.707028 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.707035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.708001 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.708005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.709220 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.709242 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.710047 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.710053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.710178 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.710805 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:07.710825 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:07.715012 (+ 4187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":273,"threads_started":1}
W20260807 19:32:07.715462 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:07.705647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:38499 (local address 127.8.216.190:33351)
0807 19:32:07.705910 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:07.705916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:07.706125 (+ 209us) server_negotiation.cc:407] Connection header received
0807 19:32:07.706257 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:07.706261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:07.706322 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:07.706401 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:07.707145 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.707870 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.709357 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.709871 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.709969 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.715212 (+ 5243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:38499: BlockingRecv error: recv got EOF from 127.8.216.131:38499 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:07 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:07.990876 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:07.980519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:34689)
0807 19:32:07.981082 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:07.981103 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:07.981271 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:07.981623 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:07.981629 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:07.982517 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:07.982748 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.982764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.984372 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.984376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.985306 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:07.985316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.986027 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.986031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:07.986158 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.986763 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:07.986780 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:07.990515 (+ 3735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":300,"threads_started":1}
W20260807 19:32:07.991019 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:07.980614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34689 (local address 127.8.216.190:33351)
0807 19:32:07.980876 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:07.980885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:07.981197 (+ 312us) server_negotiation.cc:407] Connection header received
0807 19:32:07.981403 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:07.981411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:07.981485 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:07.981593 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:07.982906 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.984237 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.985446 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:07.985863 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:07.985972 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:07.990732 (+ 4760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:34689: BlockingRecv error: recv got EOF from 127.8.216.130:34689 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:08.418483 18965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:08.404891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:60233)
0807 19:32:08.405478 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:08.405494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:08.405609 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:08.405960 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:08.405964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:08.406381 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:08.406552 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.406559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.407616 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.407620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.408604 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.408615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.409294 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.409299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.409444 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:08.414467 (+ 5023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:08.414497 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:08.418248 (+ 3751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":301,"threads_started":1}
W20260807 19:32:08.418699 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:08.404953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60233 (local address 127.8.216.190:33351)
0807 19:32:08.405134 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:08.405138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:08.405573 (+ 435us) server_negotiation.cc:407] Connection header received
0807 19:32:08.405718 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:08.405722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:08.405817 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:08.405909 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:08.406696 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.407492 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.408722 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.409161 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.409253 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:08.418464 (+ 9211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:60233: BlockingRecv error: recv got EOF from 127.8.216.129:60233 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:08.726150 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:08.716494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:48713)
0807 19:32:08.717065 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:08.717081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:08.717196 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:08.717510 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:08.717513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:08.717935 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:08.718110 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.718119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.719157 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.719160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.720108 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.720152 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.720882 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.720886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.721000 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:08.721701 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:08.721719 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:08.725892 (+ 4173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
W20260807 19:32:08.726327 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:08.716574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48713 (local address 127.8.216.190:33351)
0807 19:32:08.716754 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:08.716759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:08.717156 (+ 397us) server_negotiation.cc:407] Connection header received
0807 19:32:08.717297 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:08.717300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:08.717366 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:08.717452 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:08.718250 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.719014 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.720262 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.720700 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.720804 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:08.726098 (+ 5294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:48713: BlockingRecv error: recv got EOF from 127.8.216.131:48713 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:08 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:09.001534 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:08.991919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:34731)
0807 19:32:08.992530 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:08.992553 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:08.992673 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:08.992994 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:08.992998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:08.993411 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:08.993583 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.993591 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.994556 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.994560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.995671 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:08.995691 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.996632 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.996638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:08.996816 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:08.997574 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:08.997592 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:09.001161 (+ 3569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":333,"threads_started":1}
W20260807 19:32:09.001672 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:08.992051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34731 (local address 127.8.216.190:33351)
0807 19:32:08.992269 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:08.992274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:08.992633 (+ 359us) server_negotiation.cc:407] Connection header received
0807 19:32:08.992778 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:08.992783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:08.992847 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:08.992953 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:08.993720 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.994441 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.995824 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:08.996447 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:08.996607 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:09.001402 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:34731: BlockingRecv error: recv got EOF from 127.8.216.130:34731 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 07 19:32:09 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:09 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:09.428617 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:09.419418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:48823)
0807 19:32:09.419930 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:09.419956 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:09.420116 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:09.420529 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:09.420533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:09.420991 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:09.421152 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:09.421160 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.422179 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.422185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:09.423169 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:09.423180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.423989 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.423995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:09.424210 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:09.424801 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:09.424818 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:09.428271 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
W20260807 19:32:09.428790 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:09.419498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:48823 (local address 127.8.216.190:33351)
0807 19:32:09.419731 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:09.419739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:09.420042 (+ 303us) server_negotiation.cc:407] Connection header received
0807 19:32:09.420294 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:09.420302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:09.420371 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:09.420503 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:09.421305 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.422039 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.423321 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.423825 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.423963 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:09.428473 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:48823: BlockingRecv error: recv got EOF from 127.8.216.129:48823 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 19:32:09 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:09 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:09.736477 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:09.727346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:60287)
0807 19:32:09.728000 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:09.728022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:09.728219 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:09.728568 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:09.728571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:09.728985 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:09.729133 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:09.729142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.730134 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.730138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:09.731102 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:09.731114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.731753 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.731757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:09.731884 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:09.732522 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:09.732540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:09.736227 (+ 3687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":318,"threads_started":1}
W20260807 19:32:09.736703 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:09.727391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:60287 (local address 127.8.216.190:33351)
0807 19:32:09.727587 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:09.727592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:09.728107 (+ 515us) server_negotiation.cc:407] Connection header received
0807 19:32:09.728332 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:09.728336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:09.728419 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:09.728503 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:09.729262 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.730014 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.731229 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:09.731622 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:09.731711 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:09.736419 (+ 4708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:60287: BlockingRecv error: recv got EOF from 127.8.216.131:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:10.011571 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:10.002586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:41837)
0807 19:32:10.003127 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:10.003148 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:10.003278 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:10.003607 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:10.003610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:10.003994 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:10.004241 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.004252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.005262 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.005266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.006256 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.006268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.006950 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.006954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.007070 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.007685 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:10.007702 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:10.011289 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":299,"threads_started":1}
W20260807 19:32:10.011677 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:10.002677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:41837 (local address 127.8.216.190:33351)
0807 19:32:10.002957 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:10.002964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:10.003233 (+ 269us) server_negotiation.cc:407] Connection header received
0807 19:32:10.003384 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:10.003388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:10.003469 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:10.003555 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:10.004372 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.005136 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.006379 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.006806 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.006899 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.011472 (+ 4573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:41837: BlockingRecv error: recv got EOF from 127.8.216.130:41837 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:10.439067 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:10.429886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:39321)
0807 19:32:10.430394 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:10.430411 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:10.430543 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:10.430840 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:10.430843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:10.431260 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:10.431454 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.431462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.432576 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.432580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.433499 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.433510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.434147 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.434151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.434280 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.434902 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:10.434919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:10.438733 (+ 3814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":295,"threads_started":1}
W20260807 19:32:10.439318 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:10.430101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:39321 (local address 127.8.216.190:33351)
0807 19:32:10.430363 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:10.430372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:10.430503 (+ 131us) server_negotiation.cc:407] Connection header received
0807 19:32:10.430640 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:10.430644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:10.430705 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:10.430808 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:10.431598 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.432463 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.433624 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.434014 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.434109 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.439017 (+ 4908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:39321: BlockingRecv error: recv got EOF from 127.8.216.129:39321 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:10 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:10.746543 18973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:10.737401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:55957)
0807 19:32:10.737950 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:10.737966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:10.738081 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:10.738368 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:10.738371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:10.738768 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:10.738930 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.738938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.739971 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.739977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.741082 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:10.741096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.741787 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.741791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:10.741900 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.742541 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:10.742558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:10.746148 (+ 3590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":320,"threads_started":1}
W20260807 19:32:10.746630 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:10.737498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:55957 (local address 127.8.216.190:33351)
0807 19:32:10.737670 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:10.737674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:10.738041 (+ 367us) server_negotiation.cc:407] Connection header received
0807 19:32:10.738176 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:10.738180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:10.738242 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:10.738324 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:10.739053 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.739783 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.741229 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:10.741656 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:10.741752 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:10.746368 (+ 4616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:55957: BlockingRecv error: recv got EOF from 127.8.216.131:55957 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:11.022104 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:11.012587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:58991)
0807 19:32:11.013126 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:11.013148 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:11.013273 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:11.013719 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:11.013722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:11.014165 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:11.014355 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.014364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.015382 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.015386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.016426 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.016451 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.017493 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.017502 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.017706 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.018415 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:11.018463 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:11.021867 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":300,"threads_started":1}
W20260807 19:32:11.022336 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:11.012700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:58991 (local address 127.8.216.190:33351)
0807 19:32:11.012904 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:11.012909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:11.013227 (+ 318us) server_negotiation.cc:407] Connection header received
0807 19:32:11.013495 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:11.013500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:11.013572 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:11.013669 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:11.014504 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.015258 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.016622 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.017302 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.017438 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.022084 (+ 4646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:58991: BlockingRecv error: recv got EOF from 127.8.216.130:58991 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:11.449635 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:11.440202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:36279)
0807 19:32:11.440842 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:11.440859 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:11.440977 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:11.441285 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:11.441288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:11.441741 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:11.442017 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.442028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.443509 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.443515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.444458 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.444470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.445121 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.445126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.445251 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.445967 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:11.445984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:11.449382 (+ 3398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":339,"threads_started":1}
W20260807 19:32:11.449812 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:11.440272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36279 (local address 127.8.216.190:33351)
0807 19:32:11.440452 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:11.440457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:11.440930 (+ 473us) server_negotiation.cc:407] Connection header received
0807 19:32:11.441081 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:11.441085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:11.441145 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:11.441229 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:11.442154 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.443335 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.444579 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.444968 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.445061 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.449574 (+ 4513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:36279: BlockingRecv error: recv got EOF from 127.8.216.129:36279 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:11 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:11.758080 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:11.747633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:40691)
0807 19:32:11.748403 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:11.748428 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:11.748578 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:11.748999 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:11.749005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:11.749577 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:11.749911 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.749923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.751346 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.751354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.752602 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:11.752615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.753322 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.753328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:11.753530 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.754090 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:11.754107 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:11.757780 (+ 3673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":332,"threads_started":1}
W20260807 19:32:11.758460 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:11.748282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:40691 (local address 127.8.216.190:33351)
0807 19:32:11.748518 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:11.748525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:11.748549 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:32:11.748699 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:11.748703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:11.748796 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:11.748941 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:11.750081 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.751212 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.752741 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:11.753171 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:11.753309 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:11.758020 (+ 4711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:40691: BlockingRecv error: recv got EOF from 127.8.216.131:40691 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:12.032492 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:12.023239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:54457)
0807 19:32:12.023860 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:12.023889 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:12.024044 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:12.024477 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:12.024481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:12.024872 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:12.025041 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.025049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.026007 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.026011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.026936 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.026947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.027680 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.027684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.027814 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.028454 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:12.028471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:12.032105 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":339,"threads_started":1}
W20260807 19:32:12.032639 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:12.023270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:54457 (local address 127.8.216.190:33351)
0807 19:32:12.023549 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:12.023559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:12.023994 (+ 435us) server_negotiation.cc:407] Connection header received
0807 19:32:12.024188 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:12.024194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:12.024326 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:12.024412 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:12.025167 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.025882 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.027074 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.027524 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.027617 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.032368 (+ 4751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:54457: BlockingRecv error: recv got EOF from 127.8.216.130:54457 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:12.460497 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:12.450701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:57327)
0807 19:32:12.451492 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:12.451514 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:12.451677 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:12.452057 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:12.452061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:12.452601 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:12.452796 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.452805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.453838 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.453844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.454823 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.454834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.455569 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.455577 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.455759 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.456516 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:12.456534 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:12.460152 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":523,"threads_started":1}
W20260807 19:32:12.460647 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:12.450975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:57327 (local address 127.8.216.190:33351)
0807 19:32:12.451226 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:12.451235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:12.451609 (+ 374us) server_negotiation.cc:407] Connection header received
0807 19:32:12.451818 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:12.451824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:12.451905 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:12.452015 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:12.452945 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.453690 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.454971 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.455418 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.455553 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.460395 (+ 4842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:57327: BlockingRecv error: recv got EOF from 127.8.216.129:57327 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:12 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:12.768038 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:12.759051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:48725)
0807 19:32:12.759664 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:12.759681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:12.759800 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:12.760101 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:12.760104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:12.760613 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:12.760786 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.760794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.761748 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.761751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.762739 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:12.762750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.763426 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.763430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:12.763558 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.764212 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:12.764229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:12.767769 (+ 3540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":327,"threads_started":1}
W20260807 19:32:12.768311 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:12.759082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48725 (local address 127.8.216.190:33351)
0807 19:32:12.759311 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:12.759321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:12.759773 (+ 452us) server_negotiation.cc:407] Connection header received
0807 19:32:12.759903 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:12.759907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:12.759972 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:12.760053 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:12.760912 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.761634 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.762870 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:12.763268 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:12.763371 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:12.768013 (+ 4642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:48725: BlockingRecv error: recv got EOF from 127.8.216.131:48725 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:13.043392 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:13.033592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:51289)
0807 19:32:13.034091 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:13.034113 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:13.034235 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:13.034560 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:13.034563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:13.034999 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:13.035162 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.035170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.036260 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.036264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.037177 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.037188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.037908 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.037912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.038048 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.038722 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:13.038739 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:13.042988 (+ 4249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
W20260807 19:32:13.043491 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:13.033776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:51289 (local address 127.8.216.190:33351)
0807 19:32:13.034050 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:13.034059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:13.034191 (+ 132us) server_negotiation.cc:407] Connection header received
0807 19:32:13.034337 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:13.034341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:13.034423 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:13.034509 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:13.035315 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.036083 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.037307 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.037760 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.037878 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.043202 (+ 5324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:51289: BlockingRecv error: recv got EOF from 127.8.216.130:51289 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:13.477520 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:13.461854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:59679)
0807 19:32:13.462447 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:13.462465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:13.462626 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:13.463008 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:13.463015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:13.463437 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:13.463633 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.463642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.464742 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.464747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.465692 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.465702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.466498 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.466505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.466727 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.473021 (+ 6294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:13.473049 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:13.477257 (+ 4208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":310,"threads_started":1}
W20260807 19:32:13.477764 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:13.461942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:59679 (local address 127.8.216.190:33351)
0807 19:32:13.462212 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:13.462221 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:13.462563 (+ 342us) server_negotiation.cc:407] Connection header received
0807 19:32:13.462776 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:13.462784 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:13.462858 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:13.462968 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:13.463788 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.464583 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.465847 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.466290 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.466437 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.477506 (+ 11069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:59679: BlockingRecv error: recv got EOF from 127.8.216.129:59679 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:13 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.131@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:13.779407 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:13.769271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.131:44099)
0807 19:32:13.769866 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:13.769882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:13.770051 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:13.770445 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:13.770449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:13.770919 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:13.771077 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.771087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.772231 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.772240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.773230 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:13.773244 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.774001 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.774007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:13.774198 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.774882 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:13.774900 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:13.779013 (+ 4113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":329,"threads_started":1}
W20260807 19:32:13.779528 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:13.769350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:44099 (local address 127.8.216.190:33351)
0807 19:32:13.769702 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:13.769710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:13.769973 (+ 263us) server_negotiation.cc:407] Connection header received
0807 19:32:13.770188 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:13.770192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:13.770296 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:13.770409 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:13.771264 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.772032 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.773390 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:13.773849 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:13.773987 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:13.779252 (+ 5265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.131:44099: BlockingRecv error: recv got EOF from 127.8.216.131:44099 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
Aug 07 19:32:14 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:14 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.130@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:14.054279 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:14.044756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.130:51607)
0807 19:32:14.045311 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:14.045332 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:14.045502 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:14.045848 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:14.045853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:14.046284 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:14.046487 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:14.046496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.047521 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.047530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:14.048851 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:14.048868 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.049751 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.049756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:14.049934 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.050551 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:14.050568 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:14.054019 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":283,"threads_started":1}
W20260807 19:32:14.054577 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:14.044898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:51607 (local address 127.8.216.190:33351)
0807 19:32:14.045137 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:14.045145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:14.045419 (+ 274us) server_negotiation.cc:407] Connection header received
0807 19:32:14.045631 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:14.045637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:14.045708 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:14.045816 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:14.046630 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.047369 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.048992 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.049589 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.049740 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.054244 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.130:51607: BlockingRecv error: recv got EOF from 127.8.216.130:51607 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:32:14.475795 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18496
Aug 07 19:32:14 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:32:14 dist-test-slave-8w9v krb5kdc[17866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.216.129@KRBTEST.COM for oryx/127.8.216.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:32:14.489295 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:14.478605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33351 (local address 127.8.216.129:49977)
0807 19:32:14.479247 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:14.479262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:14.479364 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:14.479721 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:14.479724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:14.480243 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:14.480454 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:14.480486 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.481774 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.481782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:14.482914 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:14.482930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.483744 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.483751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:14.483948 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.484715 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:14.484731 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:14.488947 (+ 4216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.216.190:33351: Server oryx/127.8.216.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":395,"threads_started":1}
W20260807 19:32:14.489499 18657 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:14.478800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49977 (local address 127.8.216.190:33351)
0807 19:32:14.478994 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:14.479001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:14.479383 (+ 382us) server_negotiation.cc:407] Connection header received
0807 19:32:14.479465 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:14.479472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:14.479563 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:14.479695 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:14.480688 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.481616 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.483134 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.483588 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.483689 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.489229 (+ 5540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.216.129:49977: BlockingRecv error: recv got EOF from 127.8.216.129:49977 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:32:14.491205 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:14.477222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.8.216.129:38049)
0807 19:32:14.477905 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:14.477916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:14.477949 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:32:14.478018 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:14.478024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:14.478322 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:14.478546 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:14.480558 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.481748 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.483053 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.483700 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.483821 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.486860 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:14.486869 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:14.486884 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:14.486935 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:14.490489 (+ 3554us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:32:14.490878 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43142: Request ticket server kudu/127.8.216.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":357,"threads_started":1}
W20260807 19:32:14.491302 18987 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43142: Request ticket server kudu/127.8.216.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:32:14.491292 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18496: Not authorized: Client connection negotiation failed: client connection to 127.8.216.129:38049: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:32:14.491495 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18496
I20260807 19:32:14.513347 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18658
I20260807 19:32:14.526419 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:14.514664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57086 (local address 127.8.216.130:40165)
0807 19:32:14.515350 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:14.515357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:14.515387 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:32:14.515458 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:14.515464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:14.515710 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:14.515916 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:14.517354 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.518439 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.519708 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.520393 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.520544 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.523084 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:14.523092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:14.523106 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:14.523147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:14.525693 (+ 2546us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:32:14.526078 (+ 385us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57086: Request ticket server kudu/127.8.216.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":365,"threads_started":1}
W20260807 19:32:14.526451 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18658: Not authorized: Client connection negotiation failed: client connection to 127.8.216.130:40165: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:32:14.526547 18990 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57086: Request ticket server kudu/127.8.216.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:32:14.526649 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18658
I20260807 19:32:14.548245 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18800
I20260807 19:32:14.561904 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:14.549540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.8.216.131:34011)
0807 19:32:14.550192 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:14.550199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:14.550224 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:32:14.550298 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:14.550305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:14.550565 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:14.550725 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:14.552901 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.553962 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.555144 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:14.555697 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:14.555794 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:14.558448 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:14.558457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:14.558472 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:14.558516 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:14.561309 (+ 2793us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:32:14.561603 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49730: Request ticket server kudu/127.8.216.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":345,"threads_started":1}
W20260807 19:32:14.562036 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18800: Not authorized: Client connection negotiation failed: client connection to 127.8.216.131:34011: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.216.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:32:14.562126 18991 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49730: Request ticket server kudu/127.8.216.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:32:14.562283 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18800
I20260807 19:32:14.580492 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18422
I20260807 19:32:14.581961 18488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:14.704073 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18422
2026-08-07T19:32:14Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21996 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T19:32:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:32:14Z Disabled control of system clock
I20260807 19:32:14.805653 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35379
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:35379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:32:15.164589 18996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:15.164984 18996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:15.165175 18996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:15.173018 18996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:32:15.173179 18996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:15.173285 18996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:15.173413 18996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:32:15.173501 18996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:32:15.183264 18996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:35379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:15.184785 18996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:15.186677 18996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:15.199584 19001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:15.199679 19002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:15.199713 19004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:15.200412 18996 server_base.cc:1034] running on GCE node
I20260807 19:32:15.201232 18996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:15.202591 18996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:15.203794 18996 hybrid_clock.cc:648] HybridClock initialized: now 1786131135203749 us; error 34 us; skew 500 ppm
I20260807 19:32:15.204257 18996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:15.206637 18996 webserver.cc:460] Webserver started at http://127.8.216.190:36655/ using document root <none> and password file <none>
I20260807 19:32:15.207343 18996 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:15.207464 18996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:15.207834 18996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:15.210757 18996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "8abdf692477347b59c5c925e6ab7ffbf"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "d45616a8f4672ba9515271cd1d3ab9d3"
server_key_iv: "f1c3b402a471818035e7cc4fc45d874c"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.211495 18996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "8abdf692477347b59c5c925e6ab7ffbf"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "d45616a8f4672ba9515271cd1d3ab9d3"
server_key_iv: "f1c3b402a471818035e7cc4fc45d874c"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.216734 18996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:32:15.220116 19010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:15.221485 18996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:32:15.221678 18996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "8abdf692477347b59c5c925e6ab7ffbf"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "d45616a8f4672ba9515271cd1d3ab9d3"
server_key_iv: "f1c3b402a471818035e7cc4fc45d874c"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.221908 18996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:15.240969 18996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:15.241912 18996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:15.242215 18996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:15.261488 18996 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:35379
I20260807 19:32:15.261512 19063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:35379 every 8 connection(s)
I20260807 19:32:15.262934 18996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:32:15.266914 19064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:15.269001 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 18996
I20260807 19:32:15.269240 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:32:15.269913 9058 external_mini_cluster.cc:1428] Setting key fe7c3c82de4d01837b785be7371093f9
I20260807 19:32:15.279209 19064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf: Bootstrap starting.
I20260807 19:32:15.283170 19064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:15.284396 19064 log.cc:824] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:15.284526 19067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:15.272161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.8.216.190:35379)
0807 19:32:15.273197 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:15.273207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:15.273248 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:32:15.274922 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:15.274954 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:15.275411 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:15.275746 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:15.276447 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.277766 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.279686 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.280418 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.280551 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:15.281991 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:15.281999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:15.282013 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:15.282564 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:15.282656 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:15.282678 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:15.282767 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:15.283024 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 19:32:15.283450 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":385,"threads_started":1}
I20260807 19:32:15.287667 19064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf: No bootstrap required, opened a new log
I20260807 19:32:15.295322 19064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } }
I20260807 19:32:15.295744 19064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:15.295857 19064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abdf692477347b59c5c925e6ab7ffbf, State: Initialized, Role: FOLLOWER
I20260807 19:32:15.296530 19064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } }
I20260807 19:32:15.296810 19064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:32:15.296967 19064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:32:15.297160 19064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:15.299698 19064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } }
I20260807 19:32:15.300307 19064 leader_election.cc:302] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8abdf692477347b59c5c925e6ab7ffbf; no voters:
I20260807 19:32:15.300930 19064 leader_election.cc:288] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:32:15.301172 19069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:32:15.302062 19069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [term 1 LEADER]: Becoming Leader. State: Replica: 8abdf692477347b59c5c925e6ab7ffbf, State: Running, Role: LEADER
I20260807 19:32:15.302695 19069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } }
I20260807 19:32:15.303143 19064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:32:15.306034 19070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8abdf692477347b59c5c925e6ab7ffbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } } }
I20260807 19:32:15.306466 19070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:15.307293 19071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8abdf692477347b59c5c925e6ab7ffbf. Latest consensus state: current_term: 1 leader_uuid: "8abdf692477347b59c5c925e6ab7ffbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8abdf692477347b59c5c925e6ab7ffbf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35379 } } }
I20260807 19:32:15.307742 19071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:15.309597 19076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:32:15.314405 19076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:32:15.324105 19076 catalog_manager.cc:1347] Generated new cluster ID: 272a81c2af674825bb3e3fa2c819e1df
I20260807 19:32:15.324280 19076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:32:15.341683 19076 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:32:15.343602 19076 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:32:15.351833 19076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8abdf692477347b59c5c925e6ab7ffbf: Generated new TSK 0
I20260807 19:32:15.352886 19076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:32:15.360466 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--builtin_ntp_servers=127.8.216.148:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 19:32:15.711043 19088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:15.711545 19088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:15.711697 19088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:15.723301 19088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:15.723441 19088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:15.723609 19088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:32:15.734338 19088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:15.735694 19088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:15.737669 19088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:15.752579 19097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:15.752624 19095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:15.752957 19094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:15.754398 19088 server_base.cc:1034] running on GCE node
I20260807 19:32:15.755255 19088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:15.756690 19088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:15.757941 19088 hybrid_clock.cc:648] HybridClock initialized: now 1786131135757886 us; error 74 us; skew 500 ppm
I20260807 19:32:15.758304 19088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:15.761107 19088 webserver.cc:460] Webserver started at http://127.8.216.129:42599/ using document root <none> and password file <none>
I20260807 19:32:15.761811 19088 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:15.761916 19088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:15.762236 19088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:15.765477 19088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "e02c13364f914eddbbd943623677ba14"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "cc23796e9f3a6ac23307e59c66f6304b"
server_key_iv: "090eed448712d590980a04a1e2edac78"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.766258 19088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "e02c13364f914eddbbd943623677ba14"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "cc23796e9f3a6ac23307e59c66f6304b"
server_key_iv: "090eed448712d590980a04a1e2edac78"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.771833 19088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:32:15.775529 19104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:15.777086 19088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:32:15.777226 19088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "e02c13364f914eddbbd943623677ba14"
format_stamp: "Formatted at 2026-08-07 19:32:15 on dist-test-slave-8w9v"
server_key: "cc23796e9f3a6ac23307e59c66f6304b"
server_key_iv: "090eed448712d590980a04a1e2edac78"
server_key_version: "encryptionkey@0"
I20260807 19:32:15.777421 19088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:15.816107 19088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:15.817080 19088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:15.817442 19088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:15.818495 19088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:15.820192 19088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:15.820284 19088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:32:15.820434 19088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:15.820530 19088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:15.854982 19088 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37479
I20260807 19:32:15.855079 19222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37479 every 8 connection(s)
I20260807 19:32:15.856464 19088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:32:15.858605 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19088
I20260807 19:32:15.858856 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:32:15.859517 9058 external_mini_cluster.cc:1428] Setting key e6095344b51040e8192dcfb64cdc1a61
I20260807 19:32:15.862615 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--builtin_ntp_servers=127.8.216.148:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:32:15.869967 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:15.859378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60677 (local address 127.8.216.190:35379)
0807 19:32:15.859921 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:15.859928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:15.861405 (+ 1477us) server_negotiation.cc:407] Connection header received
0807 19:32:15.863373 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:15.863382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:15.863476 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:15.863599 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:15.864740 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.865477 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.867061 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.867469 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.867617 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:15.869069 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:15.869075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:15.869080 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:15.869106 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:15.869147 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:15.869158 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:15.869258 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:15.869501 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 19:32:15.869602 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":280,"threads_started":1}
I20260807 19:32:15.870971 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:15.860106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35379 (local address 127.8.216.129:60677)
0807 19:32:15.861238 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:15.861286 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:15.863108 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:15.863787 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:15.863795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:15.863886 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:15.864541 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:15.864560 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.865644 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.865649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:15.866901 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:15.866914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:15.867711 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:15.867718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:15.867960 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:15.868741 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:15.868744 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:15.868782 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:15.868863 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:15.869258 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:15.869272 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:15.869295 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:32:15.869476 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 19:32:15.869864 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":322,"threads_started":1}
I20260807 19:32:15.872586 19223 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35379
I20260807 19:32:15.872897 19223 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:15.873680 19223 heartbeater.cc:511] Master 127.8.216.190:35379 requested a full tablet report, sending...
I20260807 19:32:15.875901 19028 ts_manager.cc:194] Registered new tserver with Master: e02c13364f914eddbbd943623677ba14 (127.8.216.129:37479)
I20260807 19:32:15.878110 19028 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.129:60677
W20260807 19:32:16.223863 19227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:16.224249 19227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:16.224360 19227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:16.231714 19227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:16.231822 19227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:16.231931 19227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:32:16.241745 19227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:16.243170 19227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:16.245187 19227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:16.259796 19233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:16.259980 19232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:16.259900 19236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:16.261178 19227 server_base.cc:1034] running on GCE node
I20260807 19:32:16.261982 19227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:16.263218 19227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:16.264472 19227 hybrid_clock.cc:648] HybridClock initialized: now 1786131136264406 us; error 72 us; skew 500 ppm
I20260807 19:32:16.264853 19227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:16.267652 19227 webserver.cc:460] Webserver started at http://127.8.216.130:43605/ using document root <none> and password file <none>
I20260807 19:32:16.268584 19227 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:16.268697 19227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:16.269060 19227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:16.273068 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "140334c5f6cc43fa9036d9dd8c4c2f5d"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "1995e334f1252c366395809bce2a564d"
server_key_iv: "fd5c132e561235510c877bddb875311f"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.274102 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "140334c5f6cc43fa9036d9dd8c4c2f5d"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "1995e334f1252c366395809bce2a564d"
server_key_iv: "fd5c132e561235510c877bddb875311f"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.281111 19227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:32:16.285542 19242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.286988 19227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:32:16.287199 19227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "140334c5f6cc43fa9036d9dd8c4c2f5d"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "1995e334f1252c366395809bce2a564d"
server_key_iv: "fd5c132e561235510c877bddb875311f"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.287447 19227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:16.303205 19227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:16.304320 19227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:16.304646 19227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:16.305661 19227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:16.307173 19227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:16.307263 19227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.307351 19227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:16.307413 19227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.342972 19227 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:36601
I20260807 19:32:16.343046 19362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:36601 every 8 connection(s)
I20260807 19:32:16.344396 19227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:32:16.354506 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19227
I20260807 19:32:16.354714 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:32:16.355266 9058 external_mini_cluster.cc:1428] Setting key 33bfc91edb0f061c49bfaab1e4007c67
I20260807 19:32:16.357334 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.347166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:36011 (local address 127.8.216.190:35379)
0807 19:32:16.347414 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:16.347421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:16.348858 (+ 1437us) server_negotiation.cc:407] Connection header received
0807 19:32:16.350559 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:16.350563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:16.350634 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:16.350723 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:16.351737 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.352567 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.354121 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.354784 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.354962 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.356454 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:16.356461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:16.356465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:16.356490 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:16.356530 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:16.356545 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:16.356644 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:16.356853 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 19:32:16.356944 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 19:32:16.358325 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--builtin_ntp_servers=127.8.216.148:45241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:32:16.358461 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.347728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35379 (local address 127.8.216.130:36011)
0807 19:32:16.348691 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:16.348734 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:16.350318 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:16.350918 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:16.350926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:16.351005 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:16.351591 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:16.351607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.352703 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.352708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:16.353949 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:16.353963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.355026 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.355032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:16.355232 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.356047 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:16.356052 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:16.356110 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:16.356237 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:16.356648 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:16.356657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:16.356663 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:32:16.356810 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:32:16.357151 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":336,"threads_started":1}
I20260807 19:32:16.359956 19363 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35379
I20260807 19:32:16.360303 19363 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:16.361101 19363 heartbeater.cc:511] Master 127.8.216.190:35379 requested a full tablet report, sending...
I20260807 19:32:16.362514 19028 ts_manager.cc:194] Registered new tserver with Master: 140334c5f6cc43fa9036d9dd8c4c2f5d (127.8.216.130:36601)
I20260807 19:32:16.363318 19028 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.130:36011
W20260807 19:32:16.709026 19366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:16.709298 19366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:16.709383 19366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:16.716452 19366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:16.716553 19366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:16.716658 19366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:32:16.726122 19366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:45241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:16.727430 19366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:16.729277 19366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:16.743338 19374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:16.743502 19372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:16.743475 19371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:16.744463 19366 server_base.cc:1034] running on GCE node
I20260807 19:32:16.745080 19366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:16.746052 19366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:16.747277 19366 hybrid_clock.cc:648] HybridClock initialized: now 1786131136747200 us; error 66 us; skew 500 ppm
I20260807 19:32:16.747583 19366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:16.750010 19366 webserver.cc:460] Webserver started at http://127.8.216.131:35399/ using document root <none> and password file <none>
I20260807 19:32:16.750730 19366 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:16.750823 19366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:16.751108 19366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:16.754226 19366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "a73be20259d44881b5ebdf327bfd4ada"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "7377aef7991c654858ba4d1f5f760060"
server_key_iv: "cdc10d080250571474d9d195bf4f6cae"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.755039 19366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "a73be20259d44881b5ebdf327bfd4ada"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "7377aef7991c654858ba4d1f5f760060"
server_key_iv: "cdc10d080250571474d9d195bf4f6cae"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.760663 19366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260807 19:32:16.764434 19381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.765906 19366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:32:16.766129 19366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "a73be20259d44881b5ebdf327bfd4ada"
format_stamp: "Formatted at 2026-08-07 19:32:16 on dist-test-slave-8w9v"
server_key: "7377aef7991c654858ba4d1f5f760060"
server_key_iv: "cdc10d080250571474d9d195bf4f6cae"
server_key_version: "encryptionkey@0"
I20260807 19:32:16.766430 19366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:16.780431 19366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:16.781466 19366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:16.781786 19366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:16.782845 19366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:16.784543 19366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:16.784642 19366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.784782 19366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:16.784880 19366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:16.826359 19366 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:37787
I20260807 19:32:16.826469 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:37787 every 8 connection(s)
I20260807 19:32:16.827896 19366 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:32:16.838239 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19366
I20260807 19:32:16.838438 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:32:16.838927 9058 external_mini_cluster.cc:1428] Setting key 595d84ddb3364f6272906735755c2a4a
I20260807 19:32:16.841331 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.830606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:37973 (local address 127.8.216.190:35379)
0807 19:32:16.830828 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:16.830836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:16.832418 (+ 1582us) server_negotiation.cc:407] Connection header received
0807 19:32:16.834217 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:16.834225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:16.834310 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:16.834429 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:16.835455 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.836371 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.837957 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.838477 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.838636 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.840275 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:16.840280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:16.840284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:16.840304 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:16.840340 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:16.840353 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:16.840469 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:16.840737 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 19:32:16.840849 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:32:16.842350 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.831195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35379 (local address 127.8.216.131:37973)
0807 19:32:16.832234 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:16.832302 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:16.833920 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:16.834632 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:16.834639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:16.834721 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:16.835300 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:16.835317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.836540 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.836544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:16.837770 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:16.837785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.838753 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.838761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:16.839075 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.839895 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:16.839898 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:16.839937 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:16.840034 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:16.840471 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:16.840487 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:16.840498 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:32:16.840733 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 19:32:16.841131 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":322,"threads_started":1}
I20260807 19:32:16.843845 19502 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35379
I20260807 19:32:16.844187 19502 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:16.844974 19502 heartbeater.cc:511] Master 127.8.216.190:35379 requested a full tablet report, sending...
I20260807 19:32:16.846385 19028 ts_manager.cc:194] Registered new tserver with Master: a73be20259d44881b5ebdf327bfd4ada (127.8.216.131:37787)
I20260807 19:32:16.847187 19028 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.131:37973
I20260807 19:32:16.854640 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:32:16.861160 19225 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:16.859515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.8.216.190:35379)
0807 19:32:16.859746 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:16.859752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:16.860318 (+ 566us) server_negotiation.cc:407] Connection header received
0807 19:32:16.860436 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:16.860440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:16.860502 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:16.860587 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:16.860995 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50578: BlockingRecv error: recv got EOF from 127.0.0.1:50578 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:32:16.862053 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19088
I20260807 19:32:16.870263 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.863064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.8.216.129:37479)
0807 19:32:16.863602 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:16.863608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:16.863633 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:32:16.863685 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:16.863690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:16.863914 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:16.864084 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:16.864564 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.865593 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.866924 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.867478 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.867623 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.868782 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:16.868791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:16.868805 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:16.869359 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:16.869407 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:16.869426 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:16.869556 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:16.869723 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 19:32:16.869866 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":271,"threads_started":1}
I20260807 19:32:16.871634 19218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:16.959172 19223 heartbeater.cc:503] Master 127.8.216.190:35379 was elected leader, sending a full tablet report...
I20260807 19:32:16.960476 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19088
W20260807 19:32:16.980908 19013 connection.cc:535] server connection from 127.8.216.129:60677 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:32:16.981139 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19227
I20260807 19:32:16.989571 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:16.982093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.8.216.130:36601)
0807 19:32:16.982647 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:16.982655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:16.982682 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:32:16.982746 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:16.982753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:16.982981 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:16.983167 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:16.983619 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.984673 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.985946 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:16.986541 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:16.986702 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:16.987806 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:16.987814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:16.987829 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:16.988666 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:16.988719 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:16.988740 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:16.988882 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:16.989044 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 19:32:16.989182 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":264,"threads_started":1}
I20260807 19:32:16.990887 19358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:17.074466 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19227
I20260807 19:32:17.094515 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19366
I20260807 19:32:17.102984 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:17.095554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34316 (local address 127.8.216.131:37787)
0807 19:32:17.096297 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:17.096304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:17.096336 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:32:17.096404 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:17.096411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:17.096630 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:17.096763 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:17.097179 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:17.098273 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:17.099501 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:17.100091 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:17.100326 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:17.101449 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:17.101457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:17.101470 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:17.102083 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:17.102129 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:17.102153 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:17.102245 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:17.102375 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 19:32:17.102526 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":360,"threads_started":1}
I20260807 19:32:17.104506 19497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:17.187748 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19366
I20260807 19:32:17.207732 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 18996
I20260807 19:32:17.209054 19059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:17.317162 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 18996
2026-08-07T19:32:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2607 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T19:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:32:17Z Disabled control of system clock
I20260807 19:32:17.403982 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35119
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:46665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:35119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:32:17.748811 19521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:17.749231 19521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:17.749420 19521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:17.759433 19521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:32:17.759603 19521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:17.759702 19521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:17.759821 19521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:32:17.759919 19521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:32:17.770943 19521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:35119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:17.772588 19521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:17.774631 19521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:17.788574 19528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:17.788653 19529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:17.788932 19531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:17.789649 19521 server_base.cc:1034] running on GCE node
I20260807 19:32:17.790586 19521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:17.792280 19521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:17.793543 19521 hybrid_clock.cc:648] HybridClock initialized: now 1786131137793497 us; error 55 us; skew 500 ppm
I20260807 19:32:17.793958 19521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:17.796347 19521 webserver.cc:460] Webserver started at http://127.8.216.190:34861/ using document root <none> and password file <none>
I20260807 19:32:17.797089 19521 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:17.797228 19521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:17.797570 19521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:17.802608 19521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "dd74ab37a9174008bfc9319d8685f36c"
format_stamp: "Formatted at 2026-08-07 19:32:17 on dist-test-slave-8w9v"
server_key: "3be0ce15227a70759bf3d907a94bea41"
server_key_iv: "5ca4a35049524018792d960867b879ab"
server_key_version: "encryptionkey@0"
I20260807 19:32:17.803500 19521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "dd74ab37a9174008bfc9319d8685f36c"
format_stamp: "Formatted at 2026-08-07 19:32:17 on dist-test-slave-8w9v"
server_key: "3be0ce15227a70759bf3d907a94bea41"
server_key_iv: "5ca4a35049524018792d960867b879ab"
server_key_version: "encryptionkey@0"
I20260807 19:32:17.809077 19521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 19:32:17.813917 19537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:17.815459 19521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:32:17.815680 19521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "dd74ab37a9174008bfc9319d8685f36c"
format_stamp: "Formatted at 2026-08-07 19:32:17 on dist-test-slave-8w9v"
server_key: "3be0ce15227a70759bf3d907a94bea41"
server_key_iv: "5ca4a35049524018792d960867b879ab"
server_key_version: "encryptionkey@0"
I20260807 19:32:17.815958 19521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:17.822163 19521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:17.822994 19521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:17.823309 19521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:17.842382 19521 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:35119
I20260807 19:32:17.842415 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:35119 every 8 connection(s)
I20260807 19:32:17.843788 19521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:32:17.847675 19592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:17.851428 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19521
I20260807 19:32:17.851620 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:32:17.852388 9058 external_mini_cluster.cc:1428] Setting key 11cae43f08505a5fb1d9f32d8361c06b
I20260807 19:32:17.859391 19592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c: Bootstrap starting.
I20260807 19:32:17.861310 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:17.854545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.8.216.190:35119)
0807 19:32:17.855609 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:17.855620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:17.855663 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:32:17.857687 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:17.857712 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:17.858130 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:17.858460 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:17.859705 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:17.859711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:17.859723 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:17.859738 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:17.859801 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:17.859817 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:17.859995 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:17.860096 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 19:32:17.860425 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":412,"threads_started":1}
I20260807 19:32:17.862897 19592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:17.864266 19592 log.cc:824] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:17.866835 19592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c: No bootstrap required, opened a new log
I20260807 19:32:17.874468 19592 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } }
I20260807 19:32:17.874871 19592 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:17.875002 19592 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd74ab37a9174008bfc9319d8685f36c, State: Initialized, Role: FOLLOWER
I20260807 19:32:17.875578 19592 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } }
I20260807 19:32:17.875834 19592 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:32:17.876008 19592 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:32:17.876219 19592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:17.878535 19592 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } }
I20260807 19:32:17.879068 19592 leader_election.cc:302] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd74ab37a9174008bfc9319d8685f36c; no voters:
I20260807 19:32:17.879752 19592 leader_election.cc:288] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:32:17.879930 19597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:32:17.880718 19597 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [term 1 LEADER]: Becoming Leader. State: Replica: dd74ab37a9174008bfc9319d8685f36c, State: Running, Role: LEADER
I20260807 19:32:17.881268 19597 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } }
I20260807 19:32:17.882987 19592 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:32:17.884274 19598 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd74ab37a9174008bfc9319d8685f36c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } } }
I20260807 19:32:17.884373 19599 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd74ab37a9174008bfc9319d8685f36c. Latest consensus state: current_term: 1 leader_uuid: "dd74ab37a9174008bfc9319d8685f36c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd74ab37a9174008bfc9319d8685f36c" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35119 } } }
I20260807 19:32:17.884730 19598 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:17.884752 19599 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:17.887248 19602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:32:17.892899 19602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:32:17.902627 19602 catalog_manager.cc:1347] Generated new cluster ID: bb5b464a2ae245f09a14d149c8fa1e14
I20260807 19:32:17.902770 19602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:32:17.924278 19602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:32:17.924722 19602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:32:17.931988 19602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd74ab37a9174008bfc9319d8685f36c: Generated new TSK 0
I20260807 19:32:17.932847 19602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:32:17.942293 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--builtin_ntp_servers=127.8.216.148:46665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:32:18.323372 19618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:18.323659 19618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:18.323751 19618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:18.331007 19618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:18.331115 19618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:18.331223 19618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:32:18.340313 19618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:18.341558 19618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:18.343451 19618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:18.356803 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:18.356921 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:18.356882 19626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:18.357879 19618 server_base.cc:1034] running on GCE node
I20260807 19:32:18.358551 19618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:18.359441 19618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:18.360703 19618 hybrid_clock.cc:648] HybridClock initialized: now 1786131138360651 us; error 47 us; skew 500 ppm
I20260807 19:32:18.361047 19618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:18.363364 19618 webserver.cc:460] Webserver started at http://127.8.216.129:40685/ using document root <none> and password file <none>
I20260807 19:32:18.364066 19618 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:18.364236 19618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:18.364634 19618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:18.367471 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "323094d1794341d4946cad4784f8944f"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "bade9024e44cc52938af5f576ff64018"
server_key_iv: "6b0b518589247dda4a6ae40a213c88ce"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.368391 19618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "323094d1794341d4946cad4784f8944f"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "bade9024e44cc52938af5f576ff64018"
server_key_iv: "6b0b518589247dda4a6ae40a213c88ce"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.373803 19618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:32:18.377264 19632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.378672 19618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:32:18.378850 19618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "323094d1794341d4946cad4784f8944f"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "bade9024e44cc52938af5f576ff64018"
server_key_iv: "6b0b518589247dda4a6ae40a213c88ce"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.379105 19618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:18.385159 19618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:18.386004 19618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:18.386288 19618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:18.387265 19618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:18.388916 19618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:18.389050 19618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.389195 19618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:18.389287 19618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.423718 19618 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:45987
I20260807 19:32:18.423782 19750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:45987 every 8 connection(s)
I20260807 19:32:18.425166 19618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:32:18.433560 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:18.427772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:45961 (local address 127.8.216.190:35119)
0807 19:32:18.428342 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:18.428347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:18.429480 (+ 1133us) server_negotiation.cc:407] Connection header received
0807 19:32:18.431175 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:18.431181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:18.431257 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:18.431403 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:18.432780 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:18.432785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:18.432790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:18.432809 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:18.432867 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:18.432879 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:18.433015 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:18.433204 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 19:32:18.433278 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260807 19:32:18.434684 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:18.428455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35119 (local address 127.8.216.129:45961)
0807 19:32:18.429321 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:18.429369 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:18.430914 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:18.431620 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:18.431628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:18.431714 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:18.432486 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:18.432489 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:18.432525 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:18.432608 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:18.432979 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:18.432992 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:18.433002 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 19:32:18.433287 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 19:32:18.433540 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":263,"threads_started":1}
I20260807 19:32:18.434913 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19618
I20260807 19:32:18.435104 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:32:18.435817 9058 external_mini_cluster.cc:1428] Setting key 90f4ba0ece66ef031285757d45dc6a32
I20260807 19:32:18.436424 19751 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35119
I20260807 19:32:18.436756 19751 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:18.437369 19751 heartbeater.cc:511] Master 127.8.216.190:35119 requested a full tablet report, sending...
I20260807 19:32:18.438793 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--builtin_ntp_servers=127.8.216.148:46665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:32:18.439527 19555 ts_manager.cc:194] Registered new tserver with Master: 323094d1794341d4946cad4784f8944f (127.8.216.129:45987)
W20260807 19:32:18.787184 19755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:18.787485 19755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:18.787580 19755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:18.795150 19755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:18.795260 19755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:18.795382 19755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:32:18.805032 19755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:18.806391 19755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:18.808270 19755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:18.822813 19760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:18.822932 19763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:18.822815 19761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:18.823906 19755 server_base.cc:1034] running on GCE node
I20260807 19:32:18.824652 19755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:18.825646 19755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:18.826893 19755 hybrid_clock.cc:648] HybridClock initialized: now 1786131138826837 us; error 43 us; skew 500 ppm
I20260807 19:32:18.827229 19755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:18.833034 19755 webserver.cc:460] Webserver started at http://127.8.216.130:38107/ using document root <none> and password file <none>
I20260807 19:32:18.833782 19755 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:18.833931 19755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:18.834282 19755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:18.837312 19755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "727baef088da43c09fd5558f50c60f61"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "7d0068bbfb1976e6fc851c5b49b3fdab"
server_key_iv: "78438a88707226b9d80d5b2a55ca7c0c"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.838141 19755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "727baef088da43c09fd5558f50c60f61"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "7d0068bbfb1976e6fc851c5b49b3fdab"
server_key_iv: "78438a88707226b9d80d5b2a55ca7c0c"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.843624 19755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:32:18.847213 19769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.848618 19755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:32:18.848794 19755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "727baef088da43c09fd5558f50c60f61"
format_stamp: "Formatted at 2026-08-07 19:32:18 on dist-test-slave-8w9v"
server_key: "7d0068bbfb1976e6fc851c5b49b3fdab"
server_key_iv: "78438a88707226b9d80d5b2a55ca7c0c"
server_key_version: "encryptionkey@0"
I20260807 19:32:18.849066 19755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:18.854943 19755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:18.855778 19755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:18.856091 19755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:18.857146 19755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:18.858618 19755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:18.858736 19755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.858875 19755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:18.858968 19755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:18.893311 19755 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:45545
I20260807 19:32:18.893364 19887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:45545 every 8 connection(s)
I20260807 19:32:18.894821 19755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:32:18.896719 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19755
I20260807 19:32:18.896901 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:32:18.897435 9058 external_mini_cluster.cc:1428] Setting key 572a4291d1335cccd6af36716399d781
I20260807 19:32:18.900533 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--builtin_ntp_servers=127.8.216.148:46665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:32:18.903610 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:18.897586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:53121 (local address 127.8.216.190:35119)
0807 19:32:18.897809 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:18.897816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:18.899441 (+ 1625us) server_negotiation.cc:407] Connection header received
0807 19:32:18.901357 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:18.901366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:18.901434 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:18.901550 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:18.902826 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:18.902833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:18.902840 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:18.902863 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:18.902906 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:18.902924 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:18.903066 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:18.903218 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 19:32:18.903284 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:32:18.904450 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:18.898090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35119 (local address 127.8.216.130:53121)
0807 19:32:18.899243 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:18.899292 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:18.901099 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:18.901732 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:18.901739 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:18.901825 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:18.902519 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:18.902522 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:18.902566 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:18.902651 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:18.903030 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:18.903038 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:18.903043 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:32:18.903252 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 19:32:18.903451 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":374,"threads_started":1}
I20260807 19:32:18.905823 19889 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35119
I20260807 19:32:18.906133 19889 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:18.906709 19889 heartbeater.cc:511] Master 127.8.216.190:35119 requested a full tablet report, sending...
I20260807 19:32:18.908056 19555 ts_manager.cc:194] Registered new tserver with Master: 727baef088da43c09fd5558f50c60f61 (127.8.216.130:45545)
W20260807 19:32:19.254813 19892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:19.255141 19892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:19.255236 19892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:19.262884 19892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:19.263005 19892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:19.263123 19892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:32:19.272581 19892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:19.273912 19892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:19.275704 19892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:19.290338 19898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:19.290299 19897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:19.290380 19900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:19.291275 19892 server_base.cc:1034] running on GCE node
I20260807 19:32:19.291990 19892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:19.293064 19892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:19.294284 19892 hybrid_clock.cc:648] HybridClock initialized: now 1786131139294225 us; error 46 us; skew 500 ppm
I20260807 19:32:19.294690 19892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:32:19.297202 19892 webserver.cc:460] Webserver started at http://127.8.216.131:44525/ using document root <none> and password file <none>
I20260807 19:32:19.298015 19892 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:19.298143 19892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:19.298518 19892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:19.301568 19892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "286135483cf34dc3a0f5f955b4c518e9"
format_stamp: "Formatted at 2026-08-07 19:32:19 on dist-test-slave-8w9v"
server_key: "ee022ae6fa36375d69949c80db7ba90b"
server_key_iv: "66ecae287c58ff6a9f0b42c49373eb30"
server_key_version: "encryptionkey@0"
I20260807 19:32:19.302421 19892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "286135483cf34dc3a0f5f955b4c518e9"
format_stamp: "Formatted at 2026-08-07 19:32:19 on dist-test-slave-8w9v"
server_key: "ee022ae6fa36375d69949c80db7ba90b"
server_key_iv: "66ecae287c58ff6a9f0b42c49373eb30"
server_key_version: "encryptionkey@0"
I20260807 19:32:19.307771 19892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:32:19.311266 19906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:19.312706 19892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:32:19.312899 19892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "286135483cf34dc3a0f5f955b4c518e9"
format_stamp: "Formatted at 2026-08-07 19:32:19 on dist-test-slave-8w9v"
server_key: "ee022ae6fa36375d69949c80db7ba90b"
server_key_iv: "66ecae287c58ff6a9f0b42c49373eb30"
server_key_version: "encryptionkey@0"
I20260807 19:32:19.313143 19892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:19.319178 19892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:19.320015 19892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:19.320322 19892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:19.321352 19892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:19.322829 19892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:19.322957 19892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:19.323105 19892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:19.323197 19892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:19.358032 19892 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:45289
I20260807 19:32:19.358086 20026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:45289 every 8 connection(s)
I20260807 19:32:19.359890 19892 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:32:19.369455 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:19.363558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54659 (local address 127.8.216.190:35119)
0807 19:32:19.363749 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:19.363753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:19.365309 (+ 1556us) server_negotiation.cc:407] Connection header received
0807 19:32:19.367076 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:19.367082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:19.367163 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:19.367284 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:19.368646 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:19.368651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:19.368656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:19.368683 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:19.368717 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:19.368730 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:19.368861 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:19.369063 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 19:32:19.369139 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:32:19.369845 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 19892
I20260807 19:32:19.370028 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:32:19.370445 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:19.364159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35119 (local address 127.8.216.131:54659)
0807 19:32:19.365143 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:19.365178 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:19.366793 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:19.367525 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:19.367533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:19.367622 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:19.368344 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:32:19.368347 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:19.368385 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:32:19.368473 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:19.368828 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:19.368839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:19.368847 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:32:19.369146 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 19:32:19.369390 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":327,"threads_started":1}
I20260807 19:32:19.370487 9058 external_mini_cluster.cc:1428] Setting key c42800ccd01c1d7743beb6aaf1518321
I20260807 19:32:19.371866 20027 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35119
I20260807 19:32:19.372210 20027 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:19.372907 20027 heartbeater.cc:511] Master 127.8.216.190:35119 requested a full tablet report, sending...
I20260807 19:32:19.374339 19555 ts_manager.cc:194] Registered new tserver with Master: 286135483cf34dc3a0f5f955b4c518e9 (127.8.216.131:45289)
I20260807 19:32:19.386330 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:32:19.393263 19753 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:32:19.391509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.8.216.190:35119)
0807 19:32:19.391757 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:19.391766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:19.392250 (+ 484us) server_negotiation.cc:407] Connection header received
0807 19:32:19.392417 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:19.392422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:19.392490 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:19.392581 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:19.393008 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35702: BlockingRecv error: recv got EOF from 127.0.0.1:35702 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:32:19.394408 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19618
I20260807 19:32:19.398253 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:19.395367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.8.216.129:45987)
0807 19:32:19.395915 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:19.395922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:19.395952 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:32:19.396023 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:19.396029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:19.396332 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:19.396502 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:19.397520 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:19.397526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:19.397537 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:19.397555 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:19.397593 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:19.397607 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:19.397753 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:19.397859 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 19:32:19.397971 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":252,"threads_started":1}
I20260807 19:32:19.399483 19746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:19.482293 19751 heartbeater.cc:503] Master 127.8.216.190:35119 was elected leader, sending a full tablet report...
I20260807 19:32:19.483769 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19618
W20260807 19:32:19.504478 19541 connection.cc:535] server connection from 127.8.216.129:45961 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 19:32:19.504875 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19755
I20260807 19:32:19.508585 20038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:19.505867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.8.216.130:45545)
0807 19:32:19.506435 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:19.506442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:19.506471 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:32:19.506537 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:19.506543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:19.506780 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:19.506912 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:19.507808 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:19.507815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:19.507827 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:19.507847 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:19.507896 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:19.507911 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:19.508020 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:19.508097 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:32:19.508255 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":268,"threads_started":1}
I20260807 19:32:19.509835 19883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:19.592334 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19755
I20260807 19:32:19.612910 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19892
I20260807 19:32:19.617111 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:19.613921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.8.216.131:45289)
0807 19:32:19.614508 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:19.614523 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:19.614553 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:32:19.614617 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:19.614623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:19.614839 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:19.615030 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:19.616230 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:19.616252 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:19.616263 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:32:19.616276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:19.616321 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:32:19.616339 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:19.616527 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:19.616622 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:32:19.616735 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":275,"threads_started":1}
I20260807 19:32:19.618458 20022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:19.700683 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19892
I20260807 19:32:19.720319 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 19521
I20260807 19:32:19.721597 19587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:19.835853 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 19521
2026-08-07T19:32:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2518 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:32:19 dist-test-slave-8w9v krb5kdc[20045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:32:19 dist-test-slave-8w9v krb5kdc[20045](info): set up 2 sockets
Aug 07 19:32:19 dist-test-slave-8w9v krb5kdc[20045](info): commencing operation
krb5kdc: starting...
W20260807 19:32:21.102921 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:32:21 dist-test-slave-8w9v krb5kdc[20045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:32:21.576385 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41027
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:41027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:32:22.120054 20059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:22.120733 20059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:22.120995 20059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:22.133176 20059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:32:22.133471 20059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:22.133656 20059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:22.133816 20059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:32:22.133957 20059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:32:22.152446 20059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:41027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:22.155503 20059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:22.158432 20059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:22.180938 20065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:22.181061 20064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:22.182075 20070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:22.182729 20059 server_base.cc:1034] running on GCE node
I20260807 19:32:22.184088 20059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:22.186633 20059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:22.188061 20059 hybrid_clock.cc:648] HybridClock initialized: now 1786131142187996 us; error 88 us; skew 500 ppm
I20260807 19:32:22.188879 20059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:32:22 dist-test-slave-8w9v krb5kdc[20045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:32:22.196358 20059 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:32:22.199380 20059 webserver.cc:460] Webserver started at http://127.8.216.190:45515/ using document root <none> and password file <none>
I20260807 19:32:22.200871 20059 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:22.201092 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:22.201695 20059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:22.207316 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "71acca58a06a431289ea80aa52c948f8"
format_stamp: "Formatted at 2026-08-07 19:32:22 on dist-test-slave-8w9v"
server_key: "6447930bd19cf45c07def0f607d17503"
server_key_iv: "edde206b53f556868a9c09a6a65b9171"
server_key_version: "encryptionkey@0"
I20260807 19:32:22.208933 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "71acca58a06a431289ea80aa52c948f8"
format_stamp: "Formatted at 2026-08-07 19:32:22 on dist-test-slave-8w9v"
server_key: "6447930bd19cf45c07def0f607d17503"
server_key_iv: "edde206b53f556868a9c09a6a65b9171"
server_key_version: "encryptionkey@0"
I20260807 19:32:22.219138 20059 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.004s
I20260807 19:32:22.225867 20078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:22.228657 20059 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:32:22.228971 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "71acca58a06a431289ea80aa52c948f8"
format_stamp: "Formatted at 2026-08-07 19:32:22 on dist-test-slave-8w9v"
server_key: "6447930bd19cf45c07def0f607d17503"
server_key_iv: "edde206b53f556868a9c09a6a65b9171"
server_key_version: "encryptionkey@0"
I20260807 19:32:22.229377 20059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:22.254257 20059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:22.261826 20059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:22.262388 20059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:22.298610 20059 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:41027
I20260807 19:32:22.298697 20134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:41027 every 8 connection(s)
I20260807 19:32:22.301208 20059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:32:22.308446 20135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:22.311035 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 20059
I20260807 19:32:22.311373 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:32:22.312656 9058 external_mini_cluster.cc:1428] Setting key 4e6db921fbb6de762df4dadc2dfb5f29
I20260807 19:32:22.335682 20135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: Bootstrap starting.
Aug 07 19:32:22 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:32:22.342759 20135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:22.344807 20135 log.cc:824] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:22.350379 20135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: No bootstrap required, opened a new log
I20260807 19:32:22.356515 20139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:22.316260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.8.216.190:41027)
0807 19:32:22.317708 (+ 1448us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:22.317733 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:22.317820 (+ 87us) server_negotiation.cc:407] Connection header received
0807 19:32:22.320826 (+ 3006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:22.320865 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:22.321596 (+ 731us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:22.322154 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:22.325926 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:22.328053 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:22.331602 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:22.332957 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:22.333187 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:22.342073 (+ 8886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:22.342099 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:22.342127 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:22.342210 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:22.348518 (+ 6308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:22.349819 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:22.349835 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:22.349852 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:22.350162 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:22.350979 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:22.351004 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:22.351007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:22.351964 (+ 957us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:22.352571 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:22.353548 (+ 977us) server_negotiation.cc:299] Negotiation successful
0807 19:32:22.354424 (+ 876us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":707,"threads_started":1}
I20260807 19:32:22.366026 20135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } }
I20260807 19:32:22.366760 20135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:22.366938 20135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71acca58a06a431289ea80aa52c948f8, State: Initialized, Role: FOLLOWER
I20260807 19:32:22.367838 20135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } }
I20260807 19:32:22.368299 20135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:32:22.368520 20135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:32:22.368852 20135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:22.373521 20135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } }
I20260807 19:32:22.374401 20135 leader_election.cc:302] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71acca58a06a431289ea80aa52c948f8; no voters:
I20260807 19:32:22.375331 20135 leader_election.cc:288] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:32:22.375531 20141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:32:22.376966 20141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [term 1 LEADER]: Becoming Leader. State: Replica: 71acca58a06a431289ea80aa52c948f8, State: Running, Role: LEADER
I20260807 19:32:22.377875 20141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } }
I20260807 19:32:22.379264 20135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:32:22.384289 20143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71acca58a06a431289ea80aa52c948f8. Latest consensus state: current_term: 1 leader_uuid: "71acca58a06a431289ea80aa52c948f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } } }
I20260807 19:32:22.385251 20143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:22.384254 20142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71acca58a06a431289ea80aa52c948f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71acca58a06a431289ea80aa52c948f8" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41027 } } }
I20260807 19:32:22.385769 20142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:32:22.389664 20148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:32:22.400596 20148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:32:22.417582 20148 catalog_manager.cc:1347] Generated new cluster ID: 841cd796ccfc41a68de621006cd8e5bc
I20260807 19:32:22.417800 20148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:32:22.439589 20148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:32:22.442051 20148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:32:22.458673 20148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: Generated new TSK 0
I20260807 19:32:22.460402 20148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
W20260807 19:32:22.585145 9058 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.216.129: real 0.114s user 0.000s sys 0.046s
I20260807 19:32:22.587065 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:32:23.209898 20162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:23.210428 20162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:23.210670 20162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:23.224964 20162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:23.225210 20162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:23.225543 20162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:32:23.243712 20162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:23.246150 20162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:23.248752 20162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:23.279533 20169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:23.279834 20171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:23.280388 20168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:23.283787 20162 server_base.cc:1034] running on GCE node
I20260807 19:32:23.285382 20162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:23.287232 20162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:23.288749 20162 hybrid_clock.cc:648] HybridClock initialized: now 1786131143288665 us; error 75 us; skew 500 ppm
I20260807 19:32:23.289304 20162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:32:23 dist-test-slave-8w9v krb5kdc[20045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:32:23.296355 20162 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:32:23.299335 20162 webserver.cc:460] Webserver started at http://127.8.216.129:45113/ using document root <none> and password file <none>
I20260807 19:32:23.300668 20162 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:23.300920 20162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:23.301512 20162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:23.307281 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "c1f6f4e65f4e4fa99018646939ff3479"
format_stamp: "Formatted at 2026-08-07 19:32:23 on dist-test-slave-8w9v"
server_key: "0b5540aa5f408c15acc087cd835a47aa"
server_key_iv: "552a126fe6dd47e8bbcaf22628ae9735"
server_key_version: "encryptionkey@0"
I20260807 19:32:23.308816 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "c1f6f4e65f4e4fa99018646939ff3479"
format_stamp: "Formatted at 2026-08-07 19:32:23 on dist-test-slave-8w9v"
server_key: "0b5540aa5f408c15acc087cd835a47aa"
server_key_iv: "552a126fe6dd47e8bbcaf22628ae9735"
server_key_version: "encryptionkey@0"
I20260807 19:32:23.319108 20162 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.003s
I20260807 19:32:23.325414 20180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:23.328042 20162 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:32:23.328401 20162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "c1f6f4e65f4e4fa99018646939ff3479"
format_stamp: "Formatted at 2026-08-07 19:32:23 on dist-test-slave-8w9v"
server_key: "0b5540aa5f408c15acc087cd835a47aa"
server_key_iv: "552a126fe6dd47e8bbcaf22628ae9735"
server_key_version: "encryptionkey@0"
I20260807 19:32:23.328851 20162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:23.361095 20162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:23.368261 20162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:23.368871 20162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:23.370787 20162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:23.373782 20162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:23.373915 20162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:23.374238 20162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:23.374436 20162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:23.439229 20162 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:34569
I20260807 19:32:23.439308 20304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:34569 every 8 connection(s)
I20260807 19:32:23.442018 20162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:32:23.444423 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 20162
I20260807 19:32:23.444882 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:32:23.446672 9058 external_mini_cluster.cc:1428] Setting key 217f6a80756aa63f86eaade7a9706d80
Aug 07 19:32:23 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:32:23.485208 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:23.447706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:44267 (local address 127.8.216.190:41027)
0807 19:32:23.448888 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:23.448907 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:23.450793 (+ 1886us) server_negotiation.cc:407] Connection header received
0807 19:32:23.454201 (+ 3408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:23.454211 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:23.454406 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:23.454642 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:23.457573 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:23.459987 (+ 2414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:23.463720 (+ 3733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:23.465341 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:23.465645 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:23.472893 (+ 7248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:23.472908 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:23.472911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:23.472987 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:23.477963 (+ 4976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:23.480182 (+ 2219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:23.480188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:23.480191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:23.480470 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:23.481499 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:23.481503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:23.481505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:23.481991 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:23.482177 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:23.484004 (+ 1827us) server_negotiation.cc:299] Negotiation successful
0807 19:32:23.484473 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":653,"threads_started":1}
I20260807 19:32:23.488039 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:23.448918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41027 (local address 127.8.216.129:44267)
0807 19:32:23.450524 (+ 1606us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:23.450618 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:23.453783 (+ 3165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:23.455000 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:23.455013 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:23.456033 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:23.457282 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:23.457308 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:23.460333 (+ 3025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:23.460346 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:23.463429 (+ 3083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:23.463453 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:23.465741 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:23.465748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:23.466196 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:23.467383 (+ 1187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:23.467423 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:23.472381 (+ 4958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:23.478392 (+ 6011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:23.478420 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:23.478478 (+ 58us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:23.479881 (+ 1403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:23.480723 (+ 842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:23.480733 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:23.480752 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:23.481296 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:23.482213 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:23.482226 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:23.483225 (+ 999us) client_negotiation.cc:769] Sending connection context
0807 19:32:23.483935 (+ 710us) client_negotiation.cc:240] Negotiation successful
0807 19:32:23.485215 (+ 1280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"thread_start_us":688,"threads_started":1}
I20260807 19:32:23.491328 20305 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41027
I20260807 19:32:23.492038 20305 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:23.493500 20305 heartbeater.cc:511] Master 127.8.216.190:41027 requested a full tablet report, sending...
I20260807 19:32:23.497282 20096 ts_manager.cc:194] Registered new tserver with Master: c1f6f4e65f4e4fa99018646939ff3479 (127.8.216.129:34569)
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:32:23.499985 20096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:44267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
W20260807 19:32:23.552341 9058 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.216.130: real 0.101s user 0.002s sys 0.034s
I20260807 19:32:23.553526 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:32:24.084337 20311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:24.084684 20311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:24.084860 20311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:24.092664 20311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:24.092820 20311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:24.093021 20311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:32:24.103210 20311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:24.104647 20311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:24.106045 20311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:24.119644 20318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:24.119802 20321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:24.119611 20319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:24.120607 20311 server_base.cc:1034] running on GCE node
I20260807 19:32:24.121292 20311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:24.122265 20311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:24.123512 20311 hybrid_clock.cc:648] HybridClock initialized: now 1786131144123470 us; error 53 us; skew 500 ppm
I20260807 19:32:24.123845 20311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:32:24 dist-test-slave-8w9v krb5kdc[20045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:32:24.130223 20311 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:32:24.131865 20311 webserver.cc:460] Webserver started at http://127.8.216.130:33015/ using document root <none> and password file <none>
I20260807 19:32:24.132637 20311 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:24.132788 20311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:24.133107 20311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:24.136363 20311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "c58fa518472f4639a425ae73cbd92ae3"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "b776fed9d25d0fa6b6e33c3419adc140"
server_key_iv: "3034b3bd61324eea6e9f2687addebf7c"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.137380 20311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "c58fa518472f4639a425ae73cbd92ae3"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "b776fed9d25d0fa6b6e33c3419adc140"
server_key_iv: "3034b3bd61324eea6e9f2687addebf7c"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.142812 20311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:32:24.146564 20328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.147940 20311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:32:24.148171 20311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "c58fa518472f4639a425ae73cbd92ae3"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "b776fed9d25d0fa6b6e33c3419adc140"
server_key_iv: "3034b3bd61324eea6e9f2687addebf7c"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.148398 20311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:24.183049 20311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:24.187791 20311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:24.188107 20311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:24.189270 20311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:24.190791 20311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:24.190899 20311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.191061 20311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:24.191134 20311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.226358 20311 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:37513
I20260807 19:32:24.226408 20447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:37513 every 8 connection(s)
I20260807 19:32:24.227741 20311 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:32:24.229920 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 20311
I20260807 19:32:24.230116 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:32:24.230748 9058 external_mini_cluster.cc:1428] Setting key 9d5cd4f3f877258c9cc9161e3387eb6a
Aug 07 19:32:24 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:32:24.253885 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.230509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:38091 (local address 127.8.216.190:41027)
0807 19:32:24.231076 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.231083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.232149 (+ 1066us) server_negotiation.cc:407] Connection header received
0807 19:32:24.234712 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.234716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.234781 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.234874 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:24.237277 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.238233 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.240795 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.241236 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.241337 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.246144 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:24.246149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:24.246152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:24.246187 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:24.250088 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.250905 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.250914 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.250920 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.251094 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.251614 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.251622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.251627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.252043 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:24.252227 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.253116 (+ 889us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.253399 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":287,"threads_started":1}
I20260807 19:32:24.254756 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.231022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41027 (local address 127.8.216.130:38091)
0807 19:32:24.231925 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:24.231996 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:24.234398 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:24.235182 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:24.235198 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:24.236115 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:24.237076 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.237107 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.238402 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.238410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.240625 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.240644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.241486 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.241495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.241860 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.242677 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:24.242720 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:24.245867 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:24.250254 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:24.250262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:24.250276 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:24.250743 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:24.251208 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:24.251211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:24.251213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:24.251484 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:24.252215 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:24.252221 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:24.252699 (+ 478us) client_negotiation.cc:769] Sending connection context
0807 19:32:24.252990 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:32:24.253489 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":369,"threads_started":1}
I20260807 19:32:24.256469 20448 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41027
I20260807 19:32:24.256801 20448 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:24.257565 20448 heartbeater.cc:511] Master 127.8.216.190:41027 requested a full tablet report, sending...
I20260807 19:32:24.259189 20096 ts_manager.cc:194] Registered new tserver with Master: c58fa518472f4639a425ae73cbd92ae3 (127.8.216.130:37513)
I20260807 19:32:24.260593 20096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:38091
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:32:24.299890 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:32:24.505434 20305 heartbeater.cc:503] Master 127.8.216.190:41027 was elected leader, sending a full tablet report...
W20260807 19:32:24.645767 20454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:32:24.646162 20454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:32:24.646343 20454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:32:24.653764 20454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:32:24.653925 20454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:32:24.654099 20454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:32:24.663949 20454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:32:24.665329 20454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:32:24.666882 20454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:32:24.680202 20462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:24.680292 20464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:32:24.680305 20461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:32:24.681335 20454 server_base.cc:1034] running on GCE node
I20260807 19:32:24.682062 20454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:32:24.683077 20454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:32:24.684314 20454 hybrid_clock.cc:648] HybridClock initialized: now 1786131144684261 us; error 47 us; skew 500 ppm
I20260807 19:32:24.684648 20454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:32:24 dist-test-slave-8w9v krb5kdc[20045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:32:24.687868 20454 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:32:24.689477 20454 webserver.cc:460] Webserver started at http://127.8.216.131:37465/ using document root <none> and password file <none>
I20260807 19:32:24.690253 20454 fs_manager.cc:359] Metadata directory not provided
I20260807 19:32:24.690402 20454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:32:24.690711 20454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:32:24.693747 20454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "c7feab7aae984a859aa884747e0e3a28"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "e480ea8535040e5751a87d79fec88a33"
server_key_iv: "4c435907a87ff700d24b54c2bb328263"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.694551 20454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "c7feab7aae984a859aa884747e0e3a28"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "e480ea8535040e5751a87d79fec88a33"
server_key_iv: "4c435907a87ff700d24b54c2bb328263"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.699937 20454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:32:24.703403 20471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.704772 20454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:32:24.704970 20454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "c7feab7aae984a859aa884747e0e3a28"
format_stamp: "Formatted at 2026-08-07 19:32:24 on dist-test-slave-8w9v"
server_key: "e480ea8535040e5751a87d79fec88a33"
server_key_iv: "4c435907a87ff700d24b54c2bb328263"
server_key_version: "encryptionkey@0"
I20260807 19:32:24.705181 20454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:32:24.738740 20454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:32:24.742754 20454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:32:24.743067 20454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:32:24.744302 20454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:32:24.745921 20454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:32:24.746035 20454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.746170 20454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:32:24.746251 20454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:32:24.781819 20454 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:42095
I20260807 19:32:24.781863 20590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:42095 every 8 connection(s)
I20260807 19:32:24.783192 20454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:32:24.788218 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 20454
I20260807 19:32:24.788393 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:32:24.788868 9058 external_mini_cluster.cc:1428] Setting key ceaac0af1f2e247d7b825753d4e2a019
Aug 07 19:32:24 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:32:24.804345 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.785954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:33441 (local address 127.8.216.190:41027)
0807 19:32:24.786455 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.786460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.787392 (+ 932us) server_negotiation.cc:407] Connection header received
0807 19:32:24.789621 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.789628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.789732 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.789828 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:24.791754 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.792658 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.794252 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.794697 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.794858 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.798613 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:24.798619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:24.798622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:24.798658 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:24.801149 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.801889 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.801895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.801900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.802025 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.802469 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.802475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.802479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.802810 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:24.802984 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.803677 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.803923 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
I20260807 19:32:24.805348 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.786483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41027 (local address 127.8.216.131:33441)
0807 19:32:24.787198 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:24.787236 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:24.789329 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:24.790131 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:24.790147 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:24.790944 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:32:24.791561 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.791579 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.792834 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.792839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.794100 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.794113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.794960 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.794970 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.795229 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.795943 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:32:24.795983 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:32:24.798348 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:32:24.801296 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:24.801306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:24.801323 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:24.801753 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:24.802141 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:32:24.802146 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:32:24.802149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:32:24.802344 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:32:24.802943 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:32:24.802951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:32:24.803304 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 19:32:24.803595 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:32:24.804168 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":277,"threads_started":1}
I20260807 19:32:24.806845 20591 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41027
I20260807 19:32:24.807135 20591 heartbeater.cc:464] Registering TS with master...
I20260807 19:32:24.807834 20591 heartbeater.cc:511] Master 127.8.216.190:41027 requested a full tablet report, sending...
I20260807 19:32:24.809417 20096 ts_manager.cc:194] Registered new tserver with Master: c7feab7aae984a859aa884747e0e3a28 (127.8.216.131:42095)
I20260807 19:32:24.810295 20096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:33441
I20260807 19:32:24.816994 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:32:24.833962 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.821577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.8.216.190:41027)
0807 19:32:24.821803 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.821812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.822237 (+ 425us) server_negotiation.cc:407] Connection header received
0807 19:32:24.822386 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.822389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.822464 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.822544 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:24.823935 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.824752 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.825950 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.826375 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.826465 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.829019 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:24.829023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:24.829026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:24.829056 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:24.831473 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.832050 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.832053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.832056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.832245 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:24.832613 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:24.832617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:24.832618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:24.832916 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:24.833098 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.833423 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.833594 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:32:24.838690 20096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:32:24.847043 20096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:32:24.871212 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.860530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.8.216.130:37513)
0807 19:32:24.863638 (+ 3108us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.863650 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.863683 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:32:24.863813 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.863823 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.864088 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.864336 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:32:24.865060 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.866579 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.868938 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.869914 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.870072 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.870308 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.870672 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.870853 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2931,"thread_start_us":2793,"threads_started":1}
I20260807 19:32:24.871323 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.859932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:37513 (local address 127.0.0.1:47410)
0807 19:32:24.861014 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:24.861055 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:24.861226 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:24.864333 (+ 3107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:24.864338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:24.864364 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:32:24.864831 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.864844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.866694 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.866700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.868784 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.868796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.870134 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.870141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.870425 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.870470 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:32:24.870628 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:32:24.870841 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":24,"thread_start_us":625,"threads_started":1}
I20260807 19:32:24.871408 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.860075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44294)
0807 19:32:24.862024 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:24.862045 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:24.862179 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:24.862980 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:24.862985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:24.863009 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:32:24.863437 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.863452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.865495 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.865499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.868210 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.868226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.870561 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.870569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.870747 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.870782 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:32:24.870866 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:32:24.871005 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1796,"thread_start_us":4712,"threads_started":1}
I20260807 19:32:24.872289 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.861066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.8.216.129:34569)
0807 19:32:24.861752 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.861761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.862313 (+ 552us) server_negotiation.cc:407] Connection header received
0807 19:32:24.862436 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.862447 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.862728 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.862969 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:32:24.863697 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.865328 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.868436 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.870016 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.870153 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.870388 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.871752 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.871937 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":299,"threads_started":1}
I20260807 19:32:24.875433 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.861969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.8.216.131:42095)
0807 19:32:24.862550 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:24.862556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:24.865164 (+ 2608us) server_negotiation.cc:407] Connection header received
0807 19:32:24.865344 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:24.865357 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:24.865591 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:24.865812 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:32:24.866610 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.868172 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.870796 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.874007 (+ 3211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.874163 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.874440 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:24.874931 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 19:32:24.875098 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":310,"threads_started":1}
I20260807 19:32:24.875445 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:24.861417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:42095 (local address 127.0.0.1:47748)
0807 19:32:24.865042 (+ 3625us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:24.865056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:24.865195 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:24.865808 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:24.865813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:24.865842 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:32:24.866379 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.866392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.868314 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.868322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.870608 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:24.870622 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:24.874577 (+ 3955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:24.874583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:24.874745 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:24.874771 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:32:24.874884 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:32:24.875024 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3516,"mutex_wait_us":52,"thread_start_us":708,"threads_started":1}
I20260807 19:32:24.875679 20234 tablet_service.cc:1467] Processing CreateTablet for tablet 98251e908e81497486bcdca81e8c1bd4 (DEFAULT_TABLE table=test-table [id=45dfd57546cc490ab032364cae49e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:32:24.875679 20380 tablet_service.cc:1467] Processing CreateTablet for tablet 98251e908e81497486bcdca81e8c1bd4 (DEFAULT_TABLE table=test-table [id=45dfd57546cc490ab032364cae49e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:32:24.877300 20380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98251e908e81497486bcdca81e8c1bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:24.878175 20234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98251e908e81497486bcdca81e8c1bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:24.878825 20521 tablet_service.cc:1467] Processing CreateTablet for tablet 98251e908e81497486bcdca81e8c1bd4 (DEFAULT_TABLE table=test-table [id=45dfd57546cc490ab032364cae49e7db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:32:24.880081 20521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98251e908e81497486bcdca81e8c1bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:32:24.892974 20611 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Bootstrap starting.
I20260807 19:32:24.895076 20610 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Bootstrap starting.
I20260807 19:32:24.896905 20612 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Bootstrap starting.
I20260807 19:32:24.897646 20611 tablet_bootstrap.cc:654] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:24.898918 20610 tablet_bootstrap.cc:654] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:24.899181 20611 log.cc:824] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:24.900269 20610 log.cc:824] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:24.900856 20612 tablet_bootstrap.cc:654] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Neither blocks nor log segments found. Creating new log.
I20260807 19:32:24.902420 20612 log.cc:824] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Log is configured to *not* fsync() on all Append() calls
I20260807 19:32:24.906445 20611 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: No bootstrap required, opened a new log
I20260807 19:32:24.906654 20612 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: No bootstrap required, opened a new log
I20260807 19:32:24.906796 20611 ts_tablet_manager.cc:1397] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260807 19:32:24.906970 20612 ts_tablet_manager.cc:1397] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:32:24.909662 20610 tablet_bootstrap.cc:492] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: No bootstrap required, opened a new log
I20260807 19:32:24.909988 20610 ts_tablet_manager.cc:1397] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260807 19:32:24.915454 20611 raft_consensus.cc:348] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.915874 20611 raft_consensus.cc:374] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:24.915997 20611 raft_consensus.cc:729] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c58fa518472f4639a425ae73cbd92ae3, State: Initialized, Role: FOLLOWER
I20260807 19:32:24.916677 20611 consensus_queue.cc:260] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.917886 20448 heartbeater.cc:503] Master 127.8.216.190:41027 was elected leader, sending a full tablet report...
I20260807 19:32:24.917724 20612 raft_consensus.cc:348] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.918141 20612 raft_consensus.cc:374] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:24.918227 20612 raft_consensus.cc:729] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f6f4e65f4e4fa99018646939ff3479, State: Initialized, Role: FOLLOWER
I20260807 19:32:24.918567 20611 ts_tablet_manager.cc:1428] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260807 19:32:24.918809 20612 consensus_queue.cc:260] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.921041 20612 ts_tablet_manager.cc:1428] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260807 19:32:24.921130 20610 raft_consensus.cc:348] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.921608 20610 raft_consensus.cc:374] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:32:24.921726 20610 raft_consensus.cc:729] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7feab7aae984a859aa884747e0e3a28, State: Initialized, Role: FOLLOWER
I20260807 19:32:24.922338 20610 consensus_queue.cc:260] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:24.924124 20610 ts_tablet_manager.cc:1428] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Time spent starting tablet: real 0.014s user 0.005s sys 0.008s
I20260807 19:32:24.924823 20591 heartbeater.cc:503] Master 127.8.216.190:41027 was elected leader, sending a full tablet report...
W20260807 19:32:24.946130 20306 tablet.cc:2369] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:32:24.980019 20449 tablet.cc:2369] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:32:25.031095 20618 raft_consensus.cc:482] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:32:25.031507 20618 raft_consensus.cc:504] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:25.033349 20618 leader_election.cc:288] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c58fa518472f4639a425ae73cbd92ae3 (127.8.216.130:37513), c1f6f4e65f4e4fa99018646939ff3479 (127.8.216.129:34569)
W20260807 19:32:25.034947 20592 tablet.cc:2369] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:32:25.040725 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.033871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:33371 (local address 127.8.216.130:37513)
0807 19:32:25.034110 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.034117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.034278 (+ 161us) server_negotiation.cc:407] Connection header received
0807 19:32:25.034492 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.034502 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.034563 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.034689 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:32:25.035113 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.036273 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.038560 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.039840 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.040015 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.040097 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.040410 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.040519 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:32:25.040833 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.033974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:37513 (local address 127.8.216.131:33371)
0807 19:32:25.034149 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.034169 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:25.034325 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:25.034710 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:25.034715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:25.034744 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:32:25.034959 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:25.034968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.036444 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.036450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:25.038391 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:25.038406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.040005 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.040010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:25.040230 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.040266 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:32:25.040373 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:32:25.040517 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 19:32:25.041954 20401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98251e908e81497486bcdca81e8c1bd4" candidate_uuid: "c7feab7aae984a859aa884747e0e3a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58fa518472f4639a425ae73cbd92ae3" is_pre_election: true
I20260807 19:32:25.042433 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.033986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:40983 (local address 127.8.216.129:34569)
0807 19:32:25.034186 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.034192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.035743 (+ 1551us) server_negotiation.cc:407] Connection header received
0807 19:32:25.035834 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.035838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.035903 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.036033 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:32:25.036694 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.037892 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.040293 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.041459 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.041583 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.041632 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.042012 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.042109 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:32:25.042517 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.033972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.8.216.131:40983)
0807 19:32:25.035597 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.035615 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:32:25.035734 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:32:25.036045 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:32:25.036050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:32:25.036074 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:32:25.036528 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:25.036540 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.038049 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.038053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:25.040072 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:32:25.040087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.041655 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.041662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:32:25.041837 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.041867 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:32:25.041981 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:32:25.042119 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1495,"mutex_wait_us":171,"thread_start_us":458,"threads_started":1}
I20260807 19:32:25.042659 20401 raft_consensus.cc:2454] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7feab7aae984a859aa884747e0e3a28 in term 0.
I20260807 19:32:25.043632 20473 leader_election.cc:302] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c58fa518472f4639a425ae73cbd92ae3, c7feab7aae984a859aa884747e0e3a28; no voters:
I20260807 19:32:25.043635 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98251e908e81497486bcdca81e8c1bd4" candidate_uuid: "c7feab7aae984a859aa884747e0e3a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f6f4e65f4e4fa99018646939ff3479" is_pre_election: true
I20260807 19:32:25.044162 20618 raft_consensus.cc:2781] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:32:25.044246 20255 raft_consensus.cc:2454] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7feab7aae984a859aa884747e0e3a28 in term 0.
I20260807 19:32:25.044360 20618 raft_consensus.cc:482] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:32:25.044489 20618 raft_consensus.cc:3037] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:25.046947 20618 raft_consensus.cc:504] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:25.047631 20618 leader_election.cc:288] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [CANDIDATE]: Term 1 election: Requested vote from peers c58fa518472f4639a425ae73cbd92ae3 (127.8.216.130:37513), c1f6f4e65f4e4fa99018646939ff3479 (127.8.216.129:34569)
I20260807 19:32:25.048113 20255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98251e908e81497486bcdca81e8c1bd4" candidate_uuid: "c7feab7aae984a859aa884747e0e3a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1f6f4e65f4e4fa99018646939ff3479"
I20260807 19:32:25.048198 20401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98251e908e81497486bcdca81e8c1bd4" candidate_uuid: "c7feab7aae984a859aa884747e0e3a28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c58fa518472f4639a425ae73cbd92ae3"
I20260807 19:32:25.048395 20255 raft_consensus.cc:3037] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:25.048440 20401 raft_consensus.cc:3037] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:32:25.050611 20255 raft_consensus.cc:2454] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7feab7aae984a859aa884747e0e3a28 in term 1.
I20260807 19:32:25.050815 20401 raft_consensus.cc:2454] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7feab7aae984a859aa884747e0e3a28 in term 1.
I20260807 19:32:25.051028 20475 leader_election.cc:302] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1f6f4e65f4e4fa99018646939ff3479, c7feab7aae984a859aa884747e0e3a28; no voters:
I20260807 19:32:25.051359 20618 raft_consensus.cc:2781] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:32:25.051885 20618 raft_consensus.cc:686] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 1 LEADER]: Becoming Leader. State: Replica: c7feab7aae984a859aa884747e0e3a28, State: Running, Role: LEADER
I20260807 19:32:25.052572 20618 consensus_queue.cc:237] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } }
I20260807 19:32:25.057889 20094 catalog_manager.cc:5495] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7feab7aae984a859aa884747e0e3a28 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "c7feab7aae984a859aa884747e0e3a28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7feab7aae984a859aa884747e0e3a28" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 42095 } health_report { overall_health: HEALTHY } } }
I20260807 19:32:25.112798 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.107601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.8.216.131:42095)
0807 19:32:25.107790 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.107795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.107888 (+ 93us) server_negotiation.cc:407] Connection header received
0807 19:32:25.108114 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.108156 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.108258 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.108405 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:32:25.108727 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.109511 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.110820 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.111210 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.111301 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.111722 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:32:25.112330 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:32:25.112456 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.112533 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.112599 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:32:25.121181 20401 raft_consensus.cc:1261] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 1 FOLLOWER]: Refusing update from remote peer c7feab7aae984a859aa884747e0e3a28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:32:25.121181 20255 raft_consensus.cc:1261] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 1 FOLLOWER]: Refusing update from remote peer c7feab7aae984a859aa884747e0e3a28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:32:25.122198 20620 consensus_queue.cc:1035] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c58fa518472f4639a425ae73cbd92ae3" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 37513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:32:25.122530 20618 consensus_queue.cc:1035] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1f6f4e65f4e4fa99018646939ff3479" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:32:25.151861 20094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:32:25.152206 20094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:32:25.155979 20094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71acca58a06a431289ea80aa52c948f8: Sending DeleteTablet for 3 replicas of tablet 98251e908e81497486bcdca81e8c1bd4
I20260807 19:32:25.157289 20234 tablet_service.cc:1514] Processing DeleteTablet for tablet 98251e908e81497486bcdca81e8c1bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:32:25 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44294
I20260807 19:32:25.157315 20521 tablet_service.cc:1514] Processing DeleteTablet for tablet 98251e908e81497486bcdca81e8c1bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:32:25 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47748
I20260807 19:32:25.157680 20380 tablet_service.cc:1514] Processing DeleteTablet for tablet 98251e908e81497486bcdca81e8c1bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:32:25 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:47410
I20260807 19:32:25.160576 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 20162
I20260807 19:32:25.170881 20634 tablet_replica.cc:330] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: stopping tablet replica
I20260807 19:32:25.171592 20634 raft_consensus.cc:2229] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:32:25.172681 20635 tablet_replica.cc:330] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: stopping tablet replica
I20260807 19:32:25.173331 20634 raft_consensus.cc:2258] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:32:25.173915 20635 raft_consensus.cc:2229] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:32:25.174790 20635 raft_consensus.cc:2258] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:32:25.176990 20634 ts_tablet_manager.cc:1905] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:32:25.178663 20633 tablet_replica.cc:330] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: stopping tablet replica
I20260807 19:32:25.180276 20635 ts_tablet_manager.cc:1905] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:32:25.180910 20633 raft_consensus.cc:2229] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:32:25.185603 20633 raft_consensus.cc:2258] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:32:25.187042 20634 ts_tablet_manager.cc:1918] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:32:25.187232 20634 log.cc:1196] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/98251e908e81497486bcdca81e8c1bd4
I20260807 19:32:25.187877 20634 ts_tablet_manager.cc:1939] T 98251e908e81497486bcdca81e8c1bd4 P c7feab7aae984a859aa884747e0e3a28: Deleting consensus metadata
I20260807 19:32:25.189123 20633 ts_tablet_manager.cc:1905] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:32:25.190021 20080 catalog_manager.cc:4841] TS c7feab7aae984a859aa884747e0e3a28 (127.8.216.131:42095): tablet 98251e908e81497486bcdca81e8c1bd4 (table test-table [id=45dfd57546cc490ab032364cae49e7db]) successfully deleted
I20260807 19:32:25.192873 20635 ts_tablet_manager.cc:1918] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:32:25.193111 20635 log.cc:1196] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/98251e908e81497486bcdca81e8c1bd4
I20260807 19:32:25.193763 20635 ts_tablet_manager.cc:1939] T 98251e908e81497486bcdca81e8c1bd4 P c58fa518472f4639a425ae73cbd92ae3: Deleting consensus metadata
I20260807 19:32:25.195353 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.162271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44310 (local address 127.8.216.129:34569)
0807 19:32:25.162502 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.162510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.173030 (+ 10520us) server_negotiation.cc:407] Connection header received
0807 19:32:25.173403 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.173412 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.173539 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.173633 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:25.176192 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.177811 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.179792 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.180468 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.180574 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.186232 (+ 5658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:25.186254 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:25.186275 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:25.186332 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:25.191308 (+ 4976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.192228 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.192237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.192241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.192429 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.192952 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.192958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.192962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.193640 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:25.194030 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.194682 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.194958 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:32:25.195883 20080 catalog_manager.cc:4841] TS c58fa518472f4639a425ae73cbd92ae3 (127.8.216.130:37513): tablet 98251e908e81497486bcdca81e8c1bd4 (table test-table [id=45dfd57546cc490ab032364cae49e7db]) successfully deleted
I20260807 19:32:25.196712 20633 ts_tablet_manager.cc:1918] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:32:25.196897 20633 log.cc:1196] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/98251e908e81497486bcdca81e8c1bd4
I20260807 19:32:25.197309 20300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:25.305542 20633 ts_tablet_manager.cc:1939] T 98251e908e81497486bcdca81e8c1bd4 P c1f6f4e65f4e4fa99018646939ff3479: Deleting consensus metadata
I20260807 19:32:25.306236 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 20162
W20260807 19:32:25.335457 20083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:34569 (error 108)
I20260807 19:32:25.336339 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 20311
I20260807 19:32:25.337639 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.336912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44320)
0807 19:32:25.337342 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.337457 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300}
W20260807 19:32:25.337985 20083 catalog_manager.cc:4584] TS c1f6f4e65f4e4fa99018646939ff3479 (127.8.216.129:34569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98251e908e81497486bcdca81e8c1bd4: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:32:25.358660 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.337225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.8.216.130:37513)
0807 19:32:25.337425 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.337431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.337453 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:32:25.337588 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.337596 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.337700 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.337831 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:25.339140 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.339911 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.341223 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.341618 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.341753 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.352686 (+ 10933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:25.352703 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:25.352718 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:25.352756 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:25.355694 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.356595 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.356603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.356605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.356730 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.357146 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.357149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.357151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.357570 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:25.357795 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.358164 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.358341 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:32:25.359467 20442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:25.391772 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.391151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44322)
0807 19:32:25.391466 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.391583 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260807 19:32:25.393297 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.392797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44338)
0807 19:32:25.393040 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.393147 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260807 19:32:25.452735 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 20311
I20260807 19:32:25.455798 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.455245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44342)
0807 19:32:25.455456 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.455568 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:32:25.457144 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.456737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44356)
0807 19:32:25.456900 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.456989 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 19:32:25.475802 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 20454
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:32:25.492846 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.476703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.8.216.131:42095)
0807 19:32:25.476896 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:32:25.476903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:32:25.477068 (+ 165us) server_negotiation.cc:407] Connection header received
0807 19:32:25.477313 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:32:25.477317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:32:25.477439 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:32:25.477554 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:32:25.478897 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.479807 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.481215 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:32:25.481683 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:32:25.481817 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:32:25.486283 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:32:25.486301 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:32:25.486321 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:32:25.486381 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:32:25.489826 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.490620 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.490636 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.490639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.490797 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:32:25.491202 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:32:25.491207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:32:25.491210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:32:25.491598 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:32:25.491920 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 19:32:25.492269 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:32:25.492510 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:32:25.493640 20584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:25.537657 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.537021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44366)
0807 19:32:25.537291 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.537417 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 19:32:25.538815 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.538372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44380)
0807 19:32:25.538587 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.538675 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 19:32:25.619215 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 20454
I20260807 19:32:25.640573 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 20059
I20260807 19:32:25.641758 20130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:32:25.759258 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 20059
I20260807 19:32:25.759378 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:32:25.758427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34569 (local address 127.0.0.1:44396)
0807 19:32:25.759055 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 19:32:25.759163 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465}
2026-08-07T19:32:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5935 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20645](info): set up 2 sockets
Aug 07 19:32:25 dist-test-slave-8w9v krb5kdc[20645](info): commencing operation
krb5kdc: starting...
W20260807 19:32:27.024977 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.007s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:32:27 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:32:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangeradmin_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangeradmin_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangerlookup_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangerlookup_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
I20260807 19:32:27.430959 9058 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0Yp7oG/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/postgres -l logfile start
2026-08-07 19:32:30.121 UTC [20676] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:32:30.121 UTC [20676] LOG: listening on IPv4 address "127.8.216.148", port 52895
2026-08-07 19:32:30.124 UTC [20676] LOG: listening on Unix socket "/tmp/.s.PGSQL.52895"
2026-08-07 19:32:30.146 UTC [20679] LOG: database system was shut down at 2026-08-07 19:32:28 UTC
2026-08-07 19:32:30.150 UTC [20676] LOG: database system is ready to accept connections
I20260807 19:32:31.187996 9058 mini_postgres.cc:96] Postgres bound to 52895
2026-08-07 19:32:31.214 UTC [20689] FATAL: database "slave" does not exist
127.8.216.148:52895 - accepting connections
I20260807 19:32:31.215145 9058 mini_ranger.cc:161] Starting Ranger...
I20260807 19:32:31.248319 9058 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:32:31.739552 9058 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:32:31.739738 9058 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin
2026-08-07 19:32:32,262 [I] DB FLAVOR :POSTGRES
2026-08-07 19:32:32,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:32:32,263 [I] Checking connection..
2026-08-07 19:32:32,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:32:32,551 [I] Checking connection passed.
2026-08-07 19:32:32,551 [I] --------- Verifying version history table ---------
2026-08-07 19:32:32,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:32:32,819 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:32:32,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:32:33,060 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:32:33,060 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:32:33,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:32:33.320 UTC [20758] WARNING: there is no transaction in progress
2026-08-07 19:32:33,327 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:32:33,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:32:33,602 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:32:33,602 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:32:33,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:32:33,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:32:34,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-07 19:32:34,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:32:34,644 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:32:34,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:32:34,922 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:32:34,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:32:35,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:32:35,436 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:32:35,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:32:36.462 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.479 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.496 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.511 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.528 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.546 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.611 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.619 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.635 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.645 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.656 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.669 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.679 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.685 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.692 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:36.699 UTC [20872] WARNING: there is no transaction in progress
2026-08-07 19:32:37,347 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:32:37,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:32:37,606 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:32:37,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:32:37,881 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:32:37,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:32:38,133 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:32:38,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:32:38,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:32:38,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-07 19:32:38,910 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:32:38,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:32:39,175 [I] DB_PATCHES have already been applied
I20260807 19:32:39.181732 9058 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0Yp7oG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:32:39.181842 9058 mini_ranger.cc:193] Using host: 127.8.216.148
I20260807 19:32:39.202986 9058 mini_ranger.cc:227] Ranger admin URL: http://127.8.216.148:39077
Aug 07, 2026 7:32:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:32:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:32:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:32:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:32:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:32:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:32:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:32:40.436199 9058 mini_ranger.cc:160] Time spent starting Ranger: real 9.221s user 0.007s sys 0.056s
Aug 07, 2026 7:33:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:33:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 7:33:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:33:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:33:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:33:03.876055 9058 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangerkms_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/rangerkms_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.216.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.216.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:33:04 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131184, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:33:04.083446 9058 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:33:04.110991 9058 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:33:05.025816 9058 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:33:05.026054 9058 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms
I20260807 19:33:05.026362 9058 mini_ranger_kms.cc:227] Using postgres at 127.8.216.148:52895
2026-08-07 19:33:05,549 [I] DB FLAVOR :POSTGRES
2026-08-07 19:33:05,550 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:33:05,550 [I] Checking connection
2026-08-07 19:33:05,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:33:05,814 [I] connection success
2026-08-07 19:33:05,814 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:33:05,814 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:33:05,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:33:06,096 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:33:06,096 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:33:06,097 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:33:06,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:52895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:33:06,385 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:33:06.903586 9058 mini_ranger_kms.cc:310] Created kms service
I20260807 19:33:07.120482 9058 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:33:07.170182 9058 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:33:07 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131184, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:07.614075 9058 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:33:07.615882 9058 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms:/tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:33:07.615973 9058 mini_ranger_kms.cc:240] Using host: 127.8.216.148
I20260807 19:33:07.641003 9058 mini_ranger_kms.cc:276] Ranger KMS PID: 21065
I20260807 19:33:07.641218 9058 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.216.148:47215
Aug 07, 2026 7:33:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:33:08 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:33:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:33:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:33:09 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:33:09 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 19:33:09 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131189, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:33:10.057428 9058 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.974s user 0.161s sys 0.393s
I20260807 19:33:10.057827 9058 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:33:10.058051 9058 mini_ranger_kms.cc:401] 127.8.216.148:47215/kms/v1/keys
Aug 07 19:33:10 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131189, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
Aug 07 19:33:10 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131189, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
Aug 07 19:33:13 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:33:13.324067 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45389
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45389
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:47215/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:33:13.929677 21121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:33:13.930138 21121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:33:13.930368 21121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:33:13.940858 21121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:33:13.941102 21121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:33:13.941236 21121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:33:13.941373 21121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:33:13.941540 21121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:33:13.955484 21121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.216.148:47215/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45389
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:33:13.957201 21121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:33:13.958706 21121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:33:13.983500 21126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:13.983945 21129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:13.984789 21127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:13.986646 21121 server_base.cc:1034] running on GCE node
I20260807 19:33:13.987709 21121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:33:13.989473 21121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:33:13.990900 21121 hybrid_clock.cc:648] HybridClock initialized: now 1786131193990715 us; error 39 us; skew 500 ppm
I20260807 19:33:13.991573 21121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:33:13 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:33:14.002275 21121 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:33:14.004869 21121 webserver.cc:460] Webserver started at http://127.8.216.190:33047/ using document root <none> and password file <none>
I20260807 19:33:14.006032 21121 fs_manager.cc:359] Metadata directory not provided
I20260807 19:33:14.006263 21121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:33:14.006690 21121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:33:14 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:14.076535 21121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "1a7dcf410be94b5fafa19176b63d1cda"
format_stamp: "Formatted at 2026-08-07 19:33:14 on dist-test-slave-8w9v"
server_key: "8643b35fcf72d9fb0929ee9ebef27df7"
server_key_iv: "ebaf652448f2781243e21beb4cd45f99"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:14.077780 21121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "1a7dcf410be94b5fafa19176b63d1cda"
format_stamp: "Formatted at 2026-08-07 19:33:14 on dist-test-slave-8w9v"
server_key: "8643b35fcf72d9fb0929ee9ebef27df7"
server_key_iv: "ebaf652448f2781243e21beb4cd45f99"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:14.087131 21121 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.001s
I20260807 19:33:14.110364 21136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:14.112423 21121 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:33:14.112736 21121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "1a7dcf410be94b5fafa19176b63d1cda"
format_stamp: "Formatted at 2026-08-07 19:33:14 on dist-test-slave-8w9v"
server_key: "8643b35fcf72d9fb0929ee9ebef27df7"
server_key_iv: "ebaf652448f2781243e21beb4cd45f99"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:14.113023 21121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:33:14.156461 21121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:33:14.161562 21121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:33:14.161957 21121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:33:14.210983 21121 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:45389
I20260807 19:33:14.211016 21193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:45389 every 8 connection(s)
I20260807 19:33:14.212913 21121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:33:14.214546 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 21121
I20260807 19:33:14.214829 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:33:14.215953 9058 external_mini_cluster.cc:1428] Setting key ac699975e558f3d12303c4b494d857dd
I20260807 19:33:14.220713 21194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:33:14 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:33:14.238375 21194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: Bootstrap starting.
I20260807 19:33:14.243422 21194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: Neither blocks nor log segments found. Creating new log.
I20260807 19:33:14.245270 21194 log.cc:824] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: Log is configured to *not* fsync() on all Append() calls
I20260807 19:33:14.246549 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:14.218922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.8.216.190:45389)
0807 19:33:14.220850 (+ 1928us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:14.220884 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:14.220966 (+ 82us) server_negotiation.cc:407] Connection header received
0807 19:33:14.223618 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:14.223640 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:14.224097 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:14.224634 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:14.226145 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:14.228332 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:14.230375 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:14.231352 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:14.231489 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:14.236942 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:14.236952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:14.236969 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:14.237009 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:14.241406 (+ 4397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:14.242318 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:14.242329 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:14.242336 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:14.242560 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:14.243062 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:14.243068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:14.243071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:14.243880 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:14.244313 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:14.244795 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 19:33:14.245250 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1622,"thread_start_us":1193,"threads_started":1}
I20260807 19:33:14.250653 21194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: No bootstrap required, opened a new log
I20260807 19:33:14.261500 21194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } }
I20260807 19:33:14.262113 21194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:33:14.262290 21194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7dcf410be94b5fafa19176b63d1cda, State: Initialized, Role: FOLLOWER
I20260807 19:33:14.263167 21194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } }
I20260807 19:33:14.263562 21194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:33:14.263803 21194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:33:14.264211 21194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:33:14.267726 21194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } }
I20260807 19:33:14.268687 21194 leader_election.cc:302] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a7dcf410be94b5fafa19176b63d1cda; no voters:
I20260807 19:33:14.269807 21194 leader_election.cc:288] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:33:14.270028 21199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:33:14.272480 21199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [term 1 LEADER]: Becoming Leader. State: Replica: 1a7dcf410be94b5fafa19176b63d1cda, State: Running, Role: LEADER
I20260807 19:33:14.273005 21194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:33:14.273424 21199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } }
I20260807 19:33:14.279510 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a7dcf410be94b5fafa19176b63d1cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } } }
I20260807 19:33:14.280463 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [sys.catalog]: This master's current role is: LEADER
I20260807 19:33:14.279721 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a7dcf410be94b5fafa19176b63d1cda. Latest consensus state: current_term: 1 leader_uuid: "1a7dcf410be94b5fafa19176b63d1cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7dcf410be94b5fafa19176b63d1cda" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45389 } } }
I20260807 19:33:14.280826 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda [sys.catalog]: This master's current role is: LEADER
I20260807 19:33:14.292981 21206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:33:14.301725 21206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:33:14.302982 21194 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:33:14.333767 21206 catalog_manager.cc:1347] Generated new cluster ID: dcfb4ba87a1e47a2b644cdb33f63f465
I20260807 19:33:14.334086 21206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:33:14.375588 21206 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:33:14.378661 21206 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:33:14.392333 21206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: Generated new TSK 0
I20260807 19:33:14.393712 21206 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:33:15.311945 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:47215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:33:15 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131195, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:33:16.109067 21240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:33:16.109596 21240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:33:16.109840 21240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:33:16.129594 21240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:33:16.129941 21240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:33:16.130319 21240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:33:16.151908 21240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.216.148:47215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:33:16.153769 21240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:33:16.155421 21240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:33:16.186812 21252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:16.190285 21251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:16.207589 21240 server_base.cc:1034] running on GCE node
W20260807 19:33:16.209291 21254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:16.210762 21240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:33:16.225816 21240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:33:16.227262 21240 hybrid_clock.cc:648] HybridClock initialized: now 1786131196227081 us; error 120 us; skew 500 ppm
I20260807 19:33:16.227839 21240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:33:16 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:33:16.233907 21240 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:33:16.238934 21240 webserver.cc:460] Webserver started at http://127.8.216.129:33775/ using document root <none> and password file <none>
I20260807 19:33:16.240365 21240 fs_manager.cc:359] Metadata directory not provided
I20260807 19:33:16.240644 21240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:33:16.241160 21240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:33:16 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:16.265451 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "24c388f256bc4b788cbaad3522d1b985"
format_stamp: "Formatted at 2026-08-07 19:33:16 on dist-test-slave-8w9v"
server_key: "ca69278bc08c1bd279d81daf19b54428"
server_key_iv: "7db0fa4071a3dc3267535b5842c7c217"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:16.267004 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "24c388f256bc4b788cbaad3522d1b985"
format_stamp: "Formatted at 2026-08-07 19:33:16 on dist-test-slave-8w9v"
server_key: "ca69278bc08c1bd279d81daf19b54428"
server_key_iv: "7db0fa4071a3dc3267535b5842c7c217"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:16.277235 21240 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.001s
I20260807 19:33:16.329993 21264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:16.332424 21240 fs_manager.cc:725] Time spent opening block manager: real 0.030s user 0.003s sys 0.009s
I20260807 19:33:16.332772 21240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "24c388f256bc4b788cbaad3522d1b985"
format_stamp: "Formatted at 2026-08-07 19:33:16 on dist-test-slave-8w9v"
server_key: "ca69278bc08c1bd279d81daf19b54428"
server_key_iv: "7db0fa4071a3dc3267535b5842c7c217"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:16.333132 21240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 19:33:16 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131195, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:16.370306 21240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:33:16.378075 21240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:33:16.378723 21240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:33:16.380796 21240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:33:16.387877 21240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:33:16.388178 21240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20260807 19:33:16.388574 21240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:33:16.388859 21240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:16.648110 21240 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:44553
I20260807 19:33:16.648279 21382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:44553 every 8 connection(s)
I20260807 19:33:16.650532 21240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:33:16.656935 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 21240
I20260807 19:33:16.657202 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:33:16.658401 9058 external_mini_cluster.cc:1428] Setting key e0430da1eaa631f853f23785339f6e02
Aug 07 19:33:16 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:33:16.705402 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:16.659493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60495 (local address 127.8.216.190:45389)
0807 19:33:16.669108 (+ 9615us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:16.669116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:16.671493 (+ 2377us) server_negotiation.cc:407] Connection header received
0807 19:33:16.675362 (+ 3869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:16.675370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:16.675483 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:16.675578 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:16.679303 (+ 3725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:16.681023 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:16.685282 (+ 4259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:16.686661 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:16.686789 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:16.694070 (+ 7281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:16.694079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:16.694084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:16.694177 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:16.700000 (+ 5823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:16.701602 (+ 1602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:16.701609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:16.701612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:16.701895 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:16.702704 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:16.702710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:16.702713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:16.703272 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:16.703412 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:16.704654 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0807 19:33:16.704914 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9423,"thread_start_us":9230,"threads_started":1}
I20260807 19:33:16.708232 21386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:16.656045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45389 (local address 127.8.216.129:60495)
0807 19:33:16.670351 (+ 14306us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:16.670427 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:16.674816 (+ 4389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:16.676393 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:16.676407 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:16.677540 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:33:16.679008 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:16.679049 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:16.681323 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:16.681329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:16.684917 (+ 3588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:16.684956 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:16.687459 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:16.687469 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:16.687972 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:16.689241 (+ 1269us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:33:16.689279 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:33:16.693620 (+ 4341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:33:16.700366 (+ 6746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:16.700377 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:16.700398 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:16.701420 (+ 1022us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:16.702069 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:16.702074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:16.702077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:16.702526 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:16.703517 (+ 991us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:33:16.703526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:33:16.704140 (+ 614us) client_negotiation.cc:769] Sending connection context
0807 19:33:16.705788 (+ 1648us) client_negotiation.cc:240] Negotiation successful
0807 19:33:16.706465 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13892,"thread_start_us":20025,"threads_started":1}
I20260807 19:33:16.710788 21383 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45389
I20260807 19:33:16.711190 21383 heartbeater.cc:464] Registering TS with master...
I20260807 19:33:16.712783 21383 heartbeater.cc:511] Master 127.8.216.190:45389 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:33:16.716261 21156 ts_manager.cc:194] Registered new tserver with Master: 24c388f256bc4b788cbaad3522d1b985 (127.8.216.129:44553)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
I20260807 19:33:16.720464 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:60495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:33:16.764846 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:47215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:33:16 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131195, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
W20260807 19:33:17.559769 21389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:33:17.560165 21389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:33:17.560395 21389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:33:17.569994 21389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:33:17.570158 21389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:33:17.570433 21389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:33:17.583199 21389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.216.148:47215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:33:17.584728 21389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:33:17.586256 21389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:33:17.606534 21408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:17.606484 21406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:17.608327 21389 server_base.cc:1034] running on GCE node
W20260807 19:33:17.608493 21405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:17.609562 21389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:33:17.611089 21389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:33:17.612397 21389 hybrid_clock.cc:648] HybridClock initialized: now 1786131197612232 us; error 97 us; skew 500 ppm
I20260807 19:33:17.612797 21389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:33:17 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:33:17.617304 21389 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:33:17.619501 21389 webserver.cc:460] Webserver started at http://127.8.216.130:39521/ using document root <none> and password file <none>
I20260807 19:33:17.620496 21389 fs_manager.cc:359] Metadata directory not provided
I20260807 19:33:17.620677 21389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:33:17.621098 21389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:33:17 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:17.641140 21389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "1327fd321ecb40f3b3df965c76c094cc"
format_stamp: "Formatted at 2026-08-07 19:33:17 on dist-test-slave-8w9v"
server_key: "d8a60b40780634ec527bf02f597c1a54"
server_key_iv: "a66f93f0b1964259fec6da884ffc2dd8"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:17.642747 21389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "1327fd321ecb40f3b3df965c76c094cc"
format_stamp: "Formatted at 2026-08-07 19:33:17 on dist-test-slave-8w9v"
server_key: "d8a60b40780634ec527bf02f597c1a54"
server_key_iv: "a66f93f0b1964259fec6da884ffc2dd8"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:17.651528 21389 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260807 19:33:17.674194 21415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:17.676724 21389 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.002s
I20260807 19:33:17.677044 21389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "1327fd321ecb40f3b3df965c76c094cc"
format_stamp: "Formatted at 2026-08-07 19:33:17 on dist-test-slave-8w9v"
server_key: "d8a60b40780634ec527bf02f597c1a54"
server_key_iv: "a66f93f0b1964259fec6da884ffc2dd8"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:17.677508 21389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:33:17.711472 21389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:33:17.716897 21389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:33:17.717567 21389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:33:17.719363 21389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:33:17.721771 21389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:33:17.721936 21389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:17.722105 21389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:33:17.722244 21389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:17.727046 21383 heartbeater.cc:503] Master 127.8.216.190:45389 was elected leader, sending a full tablet report...
I20260807 19:33:17.840206 21389 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:45461
I20260807 19:33:17.840198 21531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:45461 every 8 connection(s)
I20260807 19:33:17.841929 21389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:33:17.850778 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 21389
I20260807 19:33:17.851001 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:33:17.851524 9058 external_mini_cluster.cc:1428] Setting key f28c216a522c1ec67851da057356307e
Aug 07 19:33:17 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131197, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:33:17.878353 21535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:17.845463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:52549 (local address 127.8.216.190:45389)
0807 19:33:17.846632 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:17.846639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:17.848049 (+ 1410us) server_negotiation.cc:407] Connection header received
0807 19:33:17.851879 (+ 3830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:17.851885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:17.851983 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:17.852145 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:17.855739 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:17.857911 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:17.860731 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:17.861619 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:17.861787 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:17.867358 (+ 5571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:17.867369 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:17.867377 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:17.867467 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:17.873084 (+ 5617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:17.874752 (+ 1668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:17.874759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:17.874762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:17.875013 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:17.875692 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:17.875697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:17.875701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:17.876150 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:17.876347 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:17.877501 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0807 19:33:17.877751 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":729,"threads_started":1}
I20260807 19:33:17.880105 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:17.846631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45389 (local address 127.8.216.130:52549)
0807 19:33:17.847831 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:17.847893 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:17.851470 (+ 3577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:17.852858 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:17.852871 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:17.854107 (+ 1236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:33:17.855505 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:17.855533 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:17.858166 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:17.858171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:17.860550 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:17.860566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:17.861947 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:17.861956 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:17.862303 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:17.863248 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:33:17.863295 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:33:17.866970 (+ 3675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:33:17.873330 (+ 6360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:17.873341 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:17.873362 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:17.874484 (+ 1122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:17.875168 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:17.875177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:17.875184 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:17.875539 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:17.876308 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:33:17.876320 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:33:17.876940 (+ 620us) client_negotiation.cc:769] Sending connection context
0807 19:33:17.877432 (+ 492us) client_negotiation.cc:240] Negotiation successful
0807 19:33:17.878161 (+ 729us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":511,"threads_started":1}
I20260807 19:33:17.882486 21532 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45389
I20260807 19:33:17.882861 21532 heartbeater.cc:464] Registering TS with master...
I20260807 19:33:17.884238 21532 heartbeater.cc:511] Master 127.8.216.190:45389 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
I20260807 19:33:17.886253 21156 ts_manager.cc:194] Registered new tserver with Master: 1327fd321ecb40f3b3df965c76c094cc (127.8.216.130:45461)
I20260807 19:33:17.887255 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:52549
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:33:17.919837 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:47215/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:33:18.525619 21539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:33:18.526002 21539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:33:18.526228 21539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:33:18.535694 21539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:33:18.535880 21539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:33:18.536041 21539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:33:18.548395 21539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.216.148:47215/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:33:18.549793 21539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:33:18.551260 21539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:33:18.569695 21544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:18.570219 21545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:33:18.570303 21548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:33:18.571764 21539 server_base.cc:1034] running on GCE node
I20260807 19:33:18.572587 21539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:33:18.573976 21539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:33:18.575282 21539 hybrid_clock.cc:648] HybridClock initialized: now 1786131198575158 us; error 69 us; skew 500 ppm
I20260807 19:33:18.575718 21539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:33:18 dist-test-slave-8w9v krb5kdc[20645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:33:18.580526 21539 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:33:18.582705 21539 webserver.cc:460] Webserver started at http://127.8.216.131:39807/ using document root <none> and password file <none>
I20260807 19:33:18.583582 21539 fs_manager.cc:359] Metadata directory not provided
I20260807 19:33:18.583916 21539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:33:18.584471 21539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:33:18 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:33:18.604442 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "33c137e7c7a144ac92dad09da3efe523"
format_stamp: "Formatted at 2026-08-07 19:33:18 on dist-test-slave-8w9v"
server_key: "00d0e1b7d5448c4d8f2f989c533b62da"
server_key_iv: "81ab0c9a5405a04caa8bee9ee0f01dfb"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:18.605871 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "33c137e7c7a144ac92dad09da3efe523"
format_stamp: "Formatted at 2026-08-07 19:33:18 on dist-test-slave-8w9v"
server_key: "00d0e1b7d5448c4d8f2f989c533b62da"
server_key_iv: "81ab0c9a5405a04caa8bee9ee0f01dfb"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:18.613480 21539 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 19:33:18.632314 21557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:18.634001 21539 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 19:33:18.634191 21539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "33c137e7c7a144ac92dad09da3efe523"
format_stamp: "Formatted at 2026-08-07 19:33:18 on dist-test-slave-8w9v"
server_key: "00d0e1b7d5448c4d8f2f989c533b62da"
server_key_iv: "81ab0c9a5405a04caa8bee9ee0f01dfb"
server_key_version: "kuduclusterkey@0"
I20260807 19:33:18.634461 21539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:33:18.671669 21539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:33:18.677565 21539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:33:18.678069 21539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:33:18.679543 21539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:33:18.681715 21539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:33:18.681830 21539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:18.682037 21539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:33:18.682123 21539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:33:18.758616 21539 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:40763
I20260807 19:33:18.758682 21679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:40763 every 8 connection(s)
I20260807 19:33:18.760229 21539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:33:18.766090 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 21539
I20260807 19:33:18.766279 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:33:18.766705 9058 external_mini_cluster.cc:1428] Setting key 2afacb9dff6ea667a505b2b6791148f0
Aug 07 19:33:18 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131198, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:33:18.788991 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.764355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:55347 (local address 127.8.216.190:45389)
0807 19:33:18.765161 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:18.765169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:18.767122 (+ 1953us) server_negotiation.cc:407] Connection header received
0807 19:33:18.770387 (+ 3265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:18.770391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:18.770470 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:18.770556 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:18.772555 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.773346 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.775569 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.776266 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.776381 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.780972 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:18.780978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:18.780982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:18.781091 (+ 109us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:18.785572 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:18.786455 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:18.786459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:18.786461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:18.786611 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:18.787079 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:18.787082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:18.787084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:18.787414 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:18.787542 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:18.788299 (+ 757us) server_negotiation.cc:299] Negotiation successful
0807 19:33:18.788554 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":454,"threads_started":1}
I20260807 19:33:18.790055 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.765346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45389 (local address 127.8.216.131:55347)
0807 19:33:18.766850 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:18.766923 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:18.770111 (+ 3188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:18.770825 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:18.770834 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:18.771508 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:33:18.772407 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.772428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.773577 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.773581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.775430 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.775455 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.776607 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.776613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.776925 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.777750 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:33:18.777774 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:33:18.780579 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:33:18.785718 (+ 5139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:18.785727 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:18.785741 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:18.786344 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:18.786703 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:33:18.786706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:33:18.786708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:33:18.786981 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:33:18.787522 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:33:18.787528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:33:18.787894 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 19:33:18.788205 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 19:33:18.788671 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":1875,"threads_started":1}
I20260807 19:33:18.791708 21680 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45389
I20260807 19:33:18.792084 21680 heartbeater.cc:464] Registering TS with master...
I20260807 19:33:18.793068 21680 heartbeater.cc:511] Master 127.8.216.190:45389 requested a full tablet report, sending...
I20260807 19:33:18.794806 21156 ts_manager.cc:194] Registered new tserver with Master: 33c137e7c7a144ac92dad09da3efe523 (127.8.216.131:40763)
I20260807 19:33:18.795971 21156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:55347
I20260807 19:33:18.796173 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:33:18.819376 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.802520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.8.216.190:45389)
0807 19:33:18.802697 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:18.802702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:18.803613 (+ 911us) server_negotiation.cc:407] Connection header received
0807 19:33:18.803848 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:18.803853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:18.803954 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:18.804035 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:18.806115 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.807211 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.808901 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.809500 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.809601 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.812989 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:18.812994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:18.812997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:18.813027 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:18.816367 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:18.817069 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:18.817072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:18.817091 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:18.817289 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:18.817764 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:18.817767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:18.817769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:18.818166 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:18.818416 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:18.818824 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:33:18.819080 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:33:18.824313 21156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:33:18.833504 21156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:33:18.870244 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.851539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59026 (local address 127.8.216.131:40763)
0807 19:33:18.857619 (+ 6080us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:18.857628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:18.858631 (+ 1003us) server_negotiation.cc:407] Connection header received
0807 19:33:18.859795 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:18.859803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:18.860061 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:18.860353 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:18.861485 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.863577 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.867295 (+ 3718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.868717 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.868883 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.869209 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:18.869465 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:33:18.869646 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5821,"thread_start_us":5989,"threads_started":1}
I20260807 19:33:18.871052 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.851240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40763 (local address 127.0.0.1:59026)
0807 19:33:18.858448 (+ 7208us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:18.858465 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:18.858724 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:18.860368 (+ 1644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:18.860375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:18.860409 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:18.861172 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.861186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.863712 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.863717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.867070 (+ 3353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.867086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.868969 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.868976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.869180 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.869237 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 19:33:18.869415 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 19:33:18.869614 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7094,"mutex_wait_us":4,"thread_start_us":1041,"threads_started":1}
I20260807 19:33:18.871821 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.852057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43276 (local address 127.8.216.130:45461)
0807 19:33:18.859640 (+ 7583us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:18.859649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:18.859677 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:33:18.859788 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:18.859797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:18.860061 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:18.861058 (+ 997us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:18.862367 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.864144 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.867851 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.870013 (+ 2162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.870304 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.870568 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:18.871084 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 19:33:18.871328 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7353,"thread_start_us":7068,"threads_started":1}
I20260807 19:33:18.871956 21694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.851119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45461 (local address 127.0.0.1:43276)
0807 19:33:18.852692 (+ 1573us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:18.852735 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:18.852946 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:18.861405 (+ 8459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:18.861410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:18.861433 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:18.862187 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.862200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.864335 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.864340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.867658 (+ 3318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.867673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.870444 (+ 2771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.870452 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.870653 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.870690 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:33:18.870785 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:33:18.870896 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"thread_start_us":7001,"threads_started":1}
I20260807 19:33:18.873066 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.858761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:44553 (local address 127.0.0.1:45858)
0807 19:33:18.860183 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:18.860224 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:18.860396 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:18.862037 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:18.862043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:18.862081 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:18.862975 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.862988 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.865252 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.865256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.868800 (+ 3544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:18.868815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.871858 (+ 3043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.871868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:18.872145 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.872183 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:33:18.872275 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:33:18.872400 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"thread_start_us":1054,"threads_started":1}
I20260807 19:33:18.873903 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:18.859311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.8.216.129:44553)
0807 19:33:18.860830 (+ 1519us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:18.860838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:18.860864 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:33:18.861013 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:18.861021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:18.861288 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:18.861561 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:18.863194 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.865091 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.869057 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:18.871087 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:18.871241 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:18.871517 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:18.872943 (+ 1426us) server_negotiation.cc:299] Negotiation successful
0807 19:33:18.873164 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1330,"thread_start_us":1164,"threads_started":1}
I20260807 19:33:18.876551 21611 tablet_service.cc:1467] Processing CreateTablet for tablet a69f7b9302934799b39f7258b5577c2b (DEFAULT_TABLE table=test-table [id=4b484b450c8147aa85cca398c7c5bdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:33:18.876725 21464 tablet_service.cc:1467] Processing CreateTablet for tablet a69f7b9302934799b39f7258b5577c2b (DEFAULT_TABLE table=test-table [id=4b484b450c8147aa85cca398c7c5bdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:33:18.878284 21611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69f7b9302934799b39f7258b5577c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:33:18.878844 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69f7b9302934799b39f7258b5577c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:33:18.880821 21315 tablet_service.cc:1467] Processing CreateTablet for tablet a69f7b9302934799b39f7258b5577c2b (DEFAULT_TABLE table=test-table [id=4b484b450c8147aa85cca398c7c5bdbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:33:18.882761 21315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69f7b9302934799b39f7258b5577c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:33:18.891247 21532 heartbeater.cc:503] Master 127.8.216.190:45389 was elected leader, sending a full tablet report...
I20260807 19:33:18.894984 21700 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Bootstrap starting.
I20260807 19:33:18.899657 21700 tablet_bootstrap.cc:654] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Neither blocks nor log segments found. Creating new log.
I20260807 19:33:18.901773 21700 log.cc:824] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Log is configured to *not* fsync() on all Append() calls
I20260807 19:33:18.904186 21704 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Bootstrap starting.
I20260807 19:33:18.905519 21702 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Bootstrap starting.
I20260807 19:33:18.906086 21700 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: No bootstrap required, opened a new log
I20260807 19:33:18.906466 21700 ts_tablet_manager.cc:1397] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 19:33:18.911154 21704 tablet_bootstrap.cc:654] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Neither blocks nor log segments found. Creating new log.
I20260807 19:33:18.911685 21702 tablet_bootstrap.cc:654] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Neither blocks nor log segments found. Creating new log.
I20260807 19:33:18.913493 21704 log.cc:824] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Log is configured to *not* fsync() on all Append() calls
I20260807 19:33:18.913861 21702 log.cc:824] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:33:18.917542 21700 raft_consensus.cc:348] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.918026 21700 raft_consensus.cc:374] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:33:18.918241 21700 raft_consensus.cc:729] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c137e7c7a144ac92dad09da3efe523, State: Initialized, Role: FOLLOWER
I20260807 19:33:18.918999 21704 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: No bootstrap required, opened a new log
I20260807 19:33:18.919111 21700 consensus_queue.cc:260] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.919423 21704 ts_tablet_manager.cc:1397] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260807 19:33:18.920782 21702 tablet_bootstrap.cc:492] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: No bootstrap required, opened a new log
I20260807 19:33:18.921242 21702 ts_tablet_manager.cc:1397] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260807 19:33:18.921257 21680 heartbeater.cc:503] Master 127.8.216.190:45389 was elected leader, sending a full tablet report...
I20260807 19:33:18.922111 21700 ts_tablet_manager.cc:1428] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Time spent starting tablet: real 0.015s user 0.007s sys 0.009s
I20260807 19:33:18.932548 21702 raft_consensus.cc:348] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.932998 21702 raft_consensus.cc:374] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:33:18.933146 21702 raft_consensus.cc:729] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1327fd321ecb40f3b3df965c76c094cc, State: Initialized, Role: FOLLOWER
I20260807 19:33:18.933786 21702 consensus_queue.cc:260] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.933792 21704 raft_consensus.cc:348] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.934190 21704 raft_consensus.cc:374] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:33:18.934482 21704 raft_consensus.cc:729] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c388f256bc4b788cbaad3522d1b985, State: Initialized, Role: FOLLOWER
I20260807 19:33:18.935418 21704 consensus_queue.cc:260] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:18.935997 21702 ts_tablet_manager.cc:1428] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260807 19:33:18.937829 21704 ts_tablet_manager.cc:1428] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Time spent starting tablet: real 0.018s user 0.003s sys 0.014s
W20260807 19:33:19.012655 21681 tablet.cc:2369] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:33:19.096872 21533 tablet.cc:2369] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:33:19.109949 21708 raft_consensus.cc:482] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:33:19.110394 21708 raft_consensus.cc:504] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:19.112928 21708 leader_election.cc:288] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33c137e7c7a144ac92dad09da3efe523 (127.8.216.131:40763), 24c388f256bc4b788cbaad3522d1b985 (127.8.216.129:44553)
I20260807 19:33:19.120991 21707 raft_consensus.cc:482] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:33:19.121498 21707 raft_consensus.cc:504] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:19.123328 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.114012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40763 (local address 127.8.216.130:51239)
0807 19:33:19.115717 (+ 1705us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:19.115746 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:19.115958 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:19.116344 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:19.116349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:19.116372 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:19.117289 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.117303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.118489 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.118497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.120548 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.120560 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.122593 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.122600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.122778 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.122807 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:33:19.122887 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:33:19.122980 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1546,"thread_start_us":1186,"threads_started":1}
I20260807 19:33:19.123471 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.113117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:51239 (local address 127.8.216.131:40763)
0807 19:33:19.113388 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.113397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.115860 (+ 2463us) server_negotiation.cc:407] Connection header received
0807 19:33:19.116147 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.116155 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.116227 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.116315 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:19.117451 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.118351 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.120747 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.122289 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.122495 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.122570 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.122965 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.123099 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:33:19.124207 21707 leader_election.cc:288] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1327fd321ecb40f3b3df965c76c094cc (127.8.216.130:45461), 24c388f256bc4b788cbaad3522d1b985 (127.8.216.129:44553)
I20260807 19:33:19.124717 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.113328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34951 (local address 127.8.216.129:44553)
0807 19:33:19.113532 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.113536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.116099 (+ 2563us) server_negotiation.cc:407] Connection header received
0807 19:33:19.116320 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.116324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.116381 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.116471 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:19.117574 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.118625 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.121280 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.122320 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.122540 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.122622 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.124286 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.124442 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:33:19.124961 21633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "1327fd321ecb40f3b3df965c76c094cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c137e7c7a144ac92dad09da3efe523" is_pre_election: true
I20260807 19:33:19.125793 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.114190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:44553 (local address 127.8.216.130:34951)
0807 19:33:19.115984 (+ 1794us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:19.115998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:19.116180 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:19.116484 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:19.116488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:19.116526 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:19.117411 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.117422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.118788 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.118792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.121078 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.121090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.123798 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.123805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.124065 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.124098 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:33:19.124979 (+ 881us) client_negotiation.cc:240] Negotiation successful
0807 19:33:19.125147 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1670,"thread_start_us":1392,"threads_started":1}
I20260807 19:33:19.125967 21633 raft_consensus.cc:2454] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1327fd321ecb40f3b3df965c76c094cc in term 0.
I20260807 19:33:19.127738 21416 leader_election.cc:302] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1327fd321ecb40f3b3df965c76c094cc, 33c137e7c7a144ac92dad09da3efe523; no voters:
I20260807 19:33:19.127889 21336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "1327fd321ecb40f3b3df965c76c094cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c388f256bc4b788cbaad3522d1b985" is_pre_election: true
I20260807 19:33:19.128540 21708 raft_consensus.cc:2781] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:33:19.128851 21708 raft_consensus.cc:482] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:33:19.128794 21336 raft_consensus.cc:2454] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1327fd321ecb40f3b3df965c76c094cc in term 0.
I20260807 19:33:19.129020 21708 raft_consensus.cc:3037] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:33:19.134161 21708 raft_consensus.cc:504] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:19.135351 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.126356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48455 (local address 127.8.216.129:44553)
0807 19:33:19.126513 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.126517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.127234 (+ 717us) server_negotiation.cc:407] Connection header received
0807 19:33:19.127523 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.127529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.127605 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.127675 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:19.129075 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.130216 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.133174 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.134210 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.134344 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.134398 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.134947 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.135076 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:33:19.135799 21633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "1327fd321ecb40f3b3df965c76c094cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c137e7c7a144ac92dad09da3efe523"
I20260807 19:33:19.135912 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.124863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45461 (local address 127.8.216.131:54251)
0807 19:33:19.125037 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:19.125055 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:19.125168 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:19.127301 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:19.127307 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:19.127399 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:19.127702 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.127716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.129662 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.129667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.132515 (+ 2848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.132531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.135002 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.135008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.135226 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.135263 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:33:19.135350 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:33:19.135519 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"spinlock_wait_cycles":24960}
I20260807 19:33:19.135895 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.125276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:44553 (local address 127.8.216.131:48455)
0807 19:33:19.127129 (+ 1853us) negotiation.cc:107] Waiting for socket to connect
0807 19:33:19.127147 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:33:19.127337 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:33:19.127912 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:33:19.127916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:33:19.127942 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:33:19.128817 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.128832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.130467 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.130470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.132939 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:33:19.132952 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.134620 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.134625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:33:19.134757 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.134780 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:33:19.134906 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:33:19.135045 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1668,"mutex_wait_us":1,"thread_start_us":7208,"threads_started":1}
I20260807 19:33:19.136075 21633 raft_consensus.cc:3037] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:33:19.136503 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.126529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:54251 (local address 127.8.216.130:45461)
0807 19:33:19.126763 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.126768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.126787 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:33:19.126854 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.126859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.126970 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.127076 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:33:19.127880 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.129464 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.132783 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.134581 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.134806 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.134878 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.135994 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.136159 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:33:19.137092 21336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "33c137e7c7a144ac92dad09da3efe523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c388f256bc4b788cbaad3522d1b985" is_pre_election: true
I20260807 19:33:19.137347 21336 raft_consensus.cc:2454] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33c137e7c7a144ac92dad09da3efe523 in term 0.
I20260807 19:33:19.137624 21486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "33c137e7c7a144ac92dad09da3efe523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1327fd321ecb40f3b3df965c76c094cc" is_pre_election: true
I20260807 19:33:19.137943 21562 leader_election.cc:302] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c388f256bc4b788cbaad3522d1b985, 33c137e7c7a144ac92dad09da3efe523; no voters:
I20260807 19:33:19.138276 21486 raft_consensus.cc:2379] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 33c137e7c7a144ac92dad09da3efe523 in current term 1: Already voted for candidate 1327fd321ecb40f3b3df965c76c094cc in this term.
I20260807 19:33:19.140484 21708 leader_election.cc:288] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [CANDIDATE]: Term 1 election: Requested vote from peers 33c137e7c7a144ac92dad09da3efe523 (127.8.216.131:40763), 24c388f256bc4b788cbaad3522d1b985 (127.8.216.129:44553)
I20260807 19:33:19.140671 21633 raft_consensus.cc:2454] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1327fd321ecb40f3b3df965c76c094cc in term 1.
I20260807 19:33:19.140926 21707 raft_consensus.cc:2741] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 19:33:19.141136 21336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69f7b9302934799b39f7258b5577c2b" candidate_uuid: "1327fd321ecb40f3b3df965c76c094cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c388f256bc4b788cbaad3522d1b985"
I20260807 19:33:19.141232 21416 leader_election.cc:302] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1327fd321ecb40f3b3df965c76c094cc, 33c137e7c7a144ac92dad09da3efe523; no voters:
I20260807 19:33:19.141353 21336 raft_consensus.cc:3037] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:33:19.141726 21708 raft_consensus.cc:2781] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:33:19.144341 21336 raft_consensus.cc:2454] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1327fd321ecb40f3b3df965c76c094cc in term 1.
I20260807 19:33:19.148602 21708 raft_consensus.cc:686] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 LEADER]: Becoming Leader. State: Replica: 1327fd321ecb40f3b3df965c76c094cc, State: Running, Role: LEADER
I20260807 19:33:19.149317 21708 consensus_queue.cc:237] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } }
I20260807 19:33:19.155074 21156 catalog_manager.cc:5495] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1327fd321ecb40f3b3df965c76c094cc (127.8.216.130). New cstate: current_term: 1 leader_uuid: "1327fd321ecb40f3b3df965c76c094cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1327fd321ecb40f3b3df965c76c094cc" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45461 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 } health_report { overall_health: UNKNOWN } } }
W20260807 19:33:19.167166 21384 tablet.cc:2369] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:33:19.233832 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.226184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.8.216.130:45461)
0807 19:33:19.226397 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.226403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.226451 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:33:19.226676 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.226681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.226779 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.226934 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:33:19.227427 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.228843 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.231110 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.231620 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.231720 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.232604 (+ 884us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:33:19.233143 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:33:19.233260 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.233445 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.233518 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:33:19.247047 21336 raft_consensus.cc:1261] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 1 FOLLOWER]: Refusing update from remote peer 1327fd321ecb40f3b3df965c76c094cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:33:19.247074 21633 raft_consensus.cc:1261] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 1 FOLLOWER]: Refusing update from remote peer 1327fd321ecb40f3b3df965c76c094cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:33:19.248075 21708 consensus_queue.cc:1035] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "33c137e7c7a144ac92dad09da3efe523" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:33:19.248504 21713 consensus_queue.cc:1035] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c388f256bc4b788cbaad3522d1b985" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 44553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:33:19.282404 21155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:33:19.283047 21155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:33:19.288461 21155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a7dcf410be94b5fafa19176b63d1cda: Sending DeleteTablet for 3 replicas of tablet a69f7b9302934799b39f7258b5577c2b
I20260807 19:33:19.290038 21611 tablet_service.cc:1514] Processing DeleteTablet for tablet a69f7b9302934799b39f7258b5577c2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:33:19 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:59026
I20260807 19:33:19.290243 21464 tablet_service.cc:1514] Processing DeleteTablet for tablet a69f7b9302934799b39f7258b5577c2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:33:19 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43276
I20260807 19:33:19.290293 21315 tablet_service.cc:1514] Processing DeleteTablet for tablet a69f7b9302934799b39f7258b5577c2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:33:19 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:45858
I20260807 19:33:19.291975 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 21240
I20260807 19:33:19.302301 21727 tablet_replica.cc:330] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: stopping tablet replica
I20260807 19:33:19.303139 21727 raft_consensus.cc:2229] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:33:19.305871 21726 tablet_replica.cc:330] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: stopping tablet replica
I20260807 19:33:19.306473 21726 raft_consensus.cc:2229] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:33:19.307060 21726 raft_consensus.cc:2258] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:33:19.307804 21727 raft_consensus.cc:2258] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:33:19.309365 21725 tablet_replica.cc:330] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: stopping tablet replica
I20260807 19:33:19.309808 21726 ts_tablet_manager.cc:1905] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:33:19.309841 21725 raft_consensus.cc:2229] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:33:19.310032 21727 ts_tablet_manager.cc:1905] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:33:19.310995 21725 raft_consensus.cc:2258] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:33:19.313431 21725 ts_tablet_manager.cc:1905] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:33:19.320019 21726 ts_tablet_manager.cc:1918] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:33:19.320429 21726 log.cc:1196] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/a69f7b9302934799b39f7258b5577c2b
I20260807 19:33:19.321313 21726 ts_tablet_manager.cc:1939] T a69f7b9302934799b39f7258b5577c2b P 33c137e7c7a144ac92dad09da3efe523: Deleting consensus metadata
Aug 07 19:33:19 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:33:19.322621 21727 ts_tablet_manager.cc:1918] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:33:19.323041 21727 log.cc:1196] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/a69f7b9302934799b39f7258b5577c2b
I20260807 19:33:19.324278 21137 catalog_manager.cc:4841] TS 33c137e7c7a144ac92dad09da3efe523 (127.8.216.131:40763): tablet a69f7b9302934799b39f7258b5577c2b (table test-table [id=4b484b450c8147aa85cca398c7c5bdbc]) successfully deleted
I20260807 19:33:19.324695 21727 ts_tablet_manager.cc:1939] T a69f7b9302934799b39f7258b5577c2b P 24c388f256bc4b788cbaad3522d1b985: Deleting consensus metadata
I20260807 19:33:19.325555 21725 ts_tablet_manager.cc:1918] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:33:19.325867 21725 log.cc:1196] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/a69f7b9302934799b39f7258b5577c2b
I20260807 19:33:19.326687 21725 ts_tablet_manager.cc:1939] T a69f7b9302934799b39f7258b5577c2b P 1327fd321ecb40f3b3df965c76c094cc: Deleting consensus metadata
I20260807 19:33:19.328822 21141 catalog_manager.cc:4841] TS 24c388f256bc4b788cbaad3522d1b985 (127.8.216.129:44553): tablet a69f7b9302934799b39f7258b5577c2b (table test-table [id=4b484b450c8147aa85cca398c7c5bdbc]) successfully deleted
I20260807 19:33:19.329195 21141 catalog_manager.cc:4841] TS 1327fd321ecb40f3b3df965c76c094cc (127.8.216.130:45461): tablet a69f7b9302934799b39f7258b5577c2b (table test-table [id=4b484b450c8147aa85cca398c7c5bdbc]) successfully deleted
I20260807 19:33:19.333376 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.294075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.8.216.129:44553)
0807 19:33:19.294313 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.294321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.299331 (+ 5010us) server_negotiation.cc:407] Connection header received
0807 19:33:19.299611 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.299620 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.299763 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.300070 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:19.302590 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.303747 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.306196 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.306885 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.306994 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.323559 (+ 16565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:19.323578 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:19.323599 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:19.323657 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:19.329523 (+ 5866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.330441 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.330455 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.330457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.330664 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.331295 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.331298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.331300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.331811 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:19.332229 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.332704 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.332916 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:33:19.334604 21377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:33:19.437327 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 21240
I20260807 19:33:19.459656 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 21389
Aug 07 19:33:19 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:33:19.479231 21697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.460632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.8.216.130:45461)
0807 19:33:19.460918 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.460924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.460953 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:33:19.461024 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.461030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.461125 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.461230 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:19.462630 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.463815 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.465147 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.465795 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.465922 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.470350 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:19.470372 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:19.470401 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:19.470447 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:19.475793 (+ 5346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.476687 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.476698 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.476702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.476889 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.477364 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.477370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.477374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.477936 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:19.478298 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.478597 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.478866 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260807 19:33:19.480077 21527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:33:19.595361 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 21389
I20260807 19:33:19.619566 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 21539
Aug 07 19:33:19 dist-test-slave-8w9v krb5kdc[20645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:33:19.634871 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:33:19.620379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.8.216.131:40763)
0807 19:33:19.620636 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:33:19.620641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:33:19.620661 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:33:19.620837 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:33:19.620841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:33:19.620923 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:33:19.621025 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:33:19.622378 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.623113 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.624572 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:33:19.624979 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:33:19.625078 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:33:19.629457 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:33:19.629472 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:33:19.629490 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:33:19.629527 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:33:19.632294 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.632934 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.632940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.632942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.633120 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:33:19.633427 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:33:19.633430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:33:19.633432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:33:19.633860 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:33:19.634080 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:33:19.634326 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 19:33:19.634488 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 19:33:19.635524 21675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:33:19.729772 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 21539
I20260807 19:33:19.753141 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 21121
I20260807 19:33:19.754410 21189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:33:19.873374 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 21121
I20260807 19:33:19.905936 9058 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:33:20.251122 9058 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:33:20.251442 9058 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:33:20.598174 9058 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:33:20.598 UTC [20676] LOG: received smart shutdown request
2026-08-07 19:33:20.602 UTC [20676] LOG: background worker "logical replication launcher" (PID 20687) exited with exit code 1
2026-08-07 19:33:20.603 UTC [20682] LOG: shutting down
2026-08-07 19:33:20.765 UTC [20676] LOG: database system is shut down
2026-08-07T19:33:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55125 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:33:21 dist-test-slave-8w9v krb5kdc[21741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:33:21 dist-test-slave-8w9v krb5kdc[21741](info): set up 2 sockets
Aug 07 19:33:21 dist-test-slave-8w9v krb5kdc[21741](info): commencing operation
krb5kdc: starting...
W20260807 19:33:22.055632 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:33:22 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:33:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:33:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangeradmin_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangeradmin_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangerlookup_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangerlookup_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
I20260807 19:33:22.435389 9058 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task0Yp7oG/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/postgres -l logfile start
2026-08-07 19:33:24.890 UTC [21772] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:33:24.890 UTC [21772] LOG: listening on IPv4 address "127.8.216.148", port 44063
2026-08-07 19:33:24.893 UTC [21772] LOG: listening on Unix socket "/tmp/.s.PGSQL.44063"
2026-08-07 19:33:24.917 UTC [21775] LOG: database system was shut down at 2026-08-07 19:33:23 UTC
2026-08-07 19:33:24.921 UTC [21772] LOG: database system is ready to accept connections
I20260807 19:33:25.994118 9058 mini_postgres.cc:96] Postgres bound to 44063
2026-08-07 19:33:26.015 UTC [21784] FATAL: database "slave" does not exist
127.8.216.148:44063 - accepting connections
I20260807 19:33:26.016513 9058 mini_ranger.cc:161] Starting Ranger...
I20260807 19:33:26.046911 9058 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:33:26.529307 9058 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:33:26.529412 9058 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin
2026-08-07 19:33:26,861 [I] DB FLAVOR :POSTGRES
2026-08-07 19:33:26,861 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:33:26,861 [I] Checking connection..
2026-08-07 19:33:26,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:33:27,129 [I] Checking connection passed.
2026-08-07 19:33:27,130 [I] --------- Verifying version history table ---------
2026-08-07 19:33:27,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:33:27,396 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:33:27,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:33:27,670 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:33:27,671 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:33:27,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:33:27.922 UTC [21852] WARNING: there is no transaction in progress
2026-08-07 19:33:27,930 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:33:27,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:33:28,192 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:33:28,192 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:33:28,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:33:28,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:33:28,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-07 19:33:28,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:33:29,244 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:33:29,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:33:29,507 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:33:29,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:33:29,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:33:30,038 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:33:30,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:33:31.058 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.075 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.093 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.112 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.129 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.151 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.211 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.218 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.233 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.244 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.254 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.270 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.280 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.288 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.295 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31.302 UTC [21967] WARNING: there is no transaction in progress
2026-08-07 19:33:31,990 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:33:31,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:33:32,273 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:33:32,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:33:32,516 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:33:32,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:33:32,785 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:33:32,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:33:33,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:33:33,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-07 19:33:33,559 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:33:33,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:33:33,828 [I] DB_PATCHES have already been applied
I20260807 19:33:33.835518 9058 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task0Yp7oG/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:33:33.835620 9058 mini_ranger.cc:193] Using host: 127.8.216.148
I20260807 19:33:33.855798 9058 mini_ranger.cc:227] Ranger admin URL: http://127.8.216.148:46431
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:33:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:33:35.038348 9058 mini_ranger.cc:160] Time spent starting Ranger: real 9.022s user 0.003s sys 0.054s
Aug 07, 2026 7:33:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:33:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 7:33:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:33:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:33:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:33:58.399552 9058 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.216.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.216.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangerkms_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.216.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/rangerkms_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.216.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.216.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.216.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/HTTP_127.8.216.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:33:58 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131238, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:33:58.609215 9058 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:33:58.637043 9058 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:33:59.632405 9058 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:33:59.632510 9058 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms
I20260807 19:33:59.632542 9058 mini_ranger_kms.cc:227] Using postgres at 127.8.216.148:44063
2026-08-07 19:34:00,014 [I] DB FLAVOR :POSTGRES
2026-08-07 19:34:00,015 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:34:00,015 [I] Checking connection
2026-08-07 19:34:00,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:34:00,265 [I] connection success
2026-08-07 19:34:00,266 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:34:00,266 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:34:00,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:34:00,533 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:34:00,533 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:34:00,534 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:34:00,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.216.148:44063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:34:00,839 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:34:01.379699 9058 mini_ranger_kms.cc:310] Created kms service
I20260807 19:34:01.569005 9058 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:34:01.615625 9058 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:34:01 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131238, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:02.046514 9058 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:34:02.046854 9058 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms:/tmp/dist-test-task0Yp7oG/build/asan/bin/postgresql.jar:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task0Yp7oG/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:34:02.046907 9058 mini_ranger_kms.cc:240] Using host: 127.8.216.148
I20260807 19:34:02.071204 9058 mini_ranger_kms.cc:276] Ranger KMS PID: 22158
I20260807 19:34:02.071357 9058 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.216.148:36625
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:34:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 19:34:04.528957 9058 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.920s user 0.102s sys 0.319s
I20260807 19:34:04.529223 9058 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:34:04.529400 9058 mini_ranger_kms.cc:401] 127.8.216.148:36625/kms/v1/keys
Aug 07 19:34:04 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131244, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 19:34:05 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131244, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
Aug 07 19:34:05 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131244, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.216.148@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
Aug 07 19:34:07 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:34:08.032765 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36089
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36089
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:08.384399 22216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:08.384768 22216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:08.384934 22216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:08.392599 22216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:08.392763 22216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:08.392863 22216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:08.392969 22216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:08.393054 22216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:08.403818 22216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.216.148:36625/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36089
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:08.405287 22216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:08.406693 22216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:08.421358 22225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:08.421437 22223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:08.421476 22222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:08.423463 22216 server_base.cc:1034] running on GCE node
I20260807 19:34:08.424487 22216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:08.426136 22216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:08.427376 22216 hybrid_clock.cc:648] HybridClock initialized: now 1786131248427320 us; error 58 us; skew 500 ppm
I20260807 19:34:08.427762 22216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:08 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:08.431329 22216 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:08.433219 22216 webserver.cc:460] Webserver started at http://127.8.216.190:39591/ using document root <none> and password file <none>
I20260807 19:34:08.433966 22216 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:08.434108 22216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:08.434448 22216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:34:08 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131248, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:08.504798 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "d8cbfa1e25ab4591986fd64d20b8e761"
format_stamp: "Formatted at 2026-08-07 19:34:08 on dist-test-slave-8w9v"
server_key: "e7063db7b8fe849ca88fe0ab182e66ee"
server_key_iv: "4c0dcce5edf16791254b984864fc7adf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:08.505825 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "d8cbfa1e25ab4591986fd64d20b8e761"
format_stamp: "Formatted at 2026-08-07 19:34:08 on dist-test-slave-8w9v"
server_key: "e7063db7b8fe849ca88fe0ab182e66ee"
server_key_iv: "4c0dcce5edf16791254b984864fc7adf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:08.512044 22216 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:34:08.531929 22232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:08.533730 22216 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:34:08.533973 22216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "d8cbfa1e25ab4591986fd64d20b8e761"
format_stamp: "Formatted at 2026-08-07 19:34:08 on dist-test-slave-8w9v"
server_key: "e7063db7b8fe849ca88fe0ab182e66ee"
server_key_iv: "4c0dcce5edf16791254b984864fc7adf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:08.534197 22216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:08.554616 22216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:08.559252 22216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:08.559631 22216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:08.580534 22216 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36089
I20260807 19:34:08.580592 22285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36089 every 8 connection(s)
I20260807 19:34:08.582080 22216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:08.586695 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22216
I20260807 19:34:08.586949 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:34:08.587121 22287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:08.587682 9058 external_mini_cluster.cc:1428] Setting key cd2c179d92d4aeb682a5ca8132044cc4
I20260807 19:34:08.603045 22287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Bootstrap starting.
I20260807 19:34:08.607635 22287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:08.609158 22287 log.cc:824] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:08.612946 22287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: No bootstrap required, opened a new log
Aug 07 19:34:08 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:08.624526 22287 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:08.625037 22287 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:08.625162 22287 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8cbfa1e25ab4591986fd64d20b8e761, State: Initialized, Role: FOLLOWER
I20260807 19:34:08.626036 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:08.590144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.8.216.190:36089)
0807 19:34:08.604706 (+ 14562us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:08.604726 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:08.604785 (+ 59us) server_negotiation.cc:407] Connection header received
0807 19:34:08.606691 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:08.606717 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:08.607144 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:08.607553 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:08.609112 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:08.610979 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:08.612476 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:08.613568 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:08.613769 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:08.617865 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:08.617872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:08.617891 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:08.617934 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:08.621331 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:08.622152 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:08.622160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:08.622167 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:08.622336 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:08.622773 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:08.622777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:08.622779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:08.623383 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:08.623726 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:08.624224 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 19:34:08.624715 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14262,"thread_start_us":14401,"threads_started":1}
I20260807 19:34:08.625886 22287 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:08.626255 22287 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:08.626434 22287 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:08.626646 22287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:08.630064 22287 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:08.630648 22287 leader_election.cc:302] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8cbfa1e25ab4591986fd64d20b8e761; no voters:
I20260807 19:34:08.631294 22287 leader_election.cc:288] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:34:08.631424 22292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:08.632881 22292 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 LEADER]: Becoming Leader. State: Replica: d8cbfa1e25ab4591986fd64d20b8e761, State: Running, Role: LEADER
I20260807 19:34:08.633574 22287 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:08.633555 22292 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:08.637122 22293 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } } }
I20260807 19:34:08.637142 22294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8cbfa1e25ab4591986fd64d20b8e761. Latest consensus state: current_term: 1 leader_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } } }
I20260807 19:34:08.637574 22294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:08.637574 22293 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:08.652576 22299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:08.658326 22299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:08.659267 22287 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:34:08.685766 22299 catalog_manager.cc:1347] Generated new cluster ID: 5cb84d36a42e4f75b00e827fd3226ef4
I20260807 19:34:08.686149 22299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:08.704531 22299 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:34:08.708271 22299 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:08.732815 22299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Generated new TSK 0
I20260807 19:34:08.734279 22299 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:34:09.696700 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:34:10.238303 22335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:10.238898 22335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:10.239125 22335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:10.249022 22335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:10.249194 22335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:10.249426 22335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:10.260352 22335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:10.261967 22335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:10.263497 22335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:10.281885 22345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:10.285655 22347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:10.286654 22349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:10.287111 22335 server_base.cc:1034] running on GCE node
I20260807 19:34:10.287814 22335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:10.289067 22335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:10.290402 22335 hybrid_clock.cc:648] HybridClock initialized: now 1786131250290302 us; error 99 us; skew 500 ppm
I20260807 19:34:10.290835 22335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:10.294332 22335 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:10.300575 22335 webserver.cc:460] Webserver started at http://127.8.216.129:38173/ using document root <none> and password file <none>
I20260807 19:34:10.301445 22335 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:10.301610 22335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:10.301967 22335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:10.332772 22335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "8da3648960274cdaa4ec0bffe520407f"
format_stamp: "Formatted at 2026-08-07 19:34:10 on dist-test-slave-8w9v"
server_key: "fa7a97fdf2d63e340498e5d3ae1d29e8"
server_key_iv: "d9de7eb5b565c6de69566c6c08f313bf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:10.334036 22335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "8da3648960274cdaa4ec0bffe520407f"
format_stamp: "Formatted at 2026-08-07 19:34:10 on dist-test-slave-8w9v"
server_key: "fa7a97fdf2d63e340498e5d3ae1d29e8"
server_key_iv: "d9de7eb5b565c6de69566c6c08f313bf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:10.355746 22335 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.012s sys 0.008s
I20260807 19:34:10.375231 22356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:10.377209 22335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:10.377437 22335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "8da3648960274cdaa4ec0bffe520407f"
format_stamp: "Formatted at 2026-08-07 19:34:10 on dist-test-slave-8w9v"
server_key: "fa7a97fdf2d63e340498e5d3ae1d29e8"
server_key_iv: "d9de7eb5b565c6de69566c6c08f313bf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:10.377832 22335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:10.431365 22335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:10.436939 22335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:10.437299 22335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:10.438657 22335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:10.440826 22335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:10.441045 22335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 19:34:10.441249 22335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:10.441377 22335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:10.500213 22335 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:46623
I20260807 19:34:10.500262 22482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:46623 every 8 connection(s)
I20260807 19:34:10.502074 22335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:10.510553 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22335
I20260807 19:34:10.510797 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:34:10.511663 9058 external_mini_cluster.cc:1428] Setting key d050bdd7d8fc141e2eb2cff9843703c2
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
I20260807 19:34:10.564289 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:10.525924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:42075 (local address 127.8.216.190:36089)
0807 19:34:10.542579 (+ 16655us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:10.542586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:10.542610 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:34:10.542678 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:10.542682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:10.542804 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:10.542934 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:10.545621 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:10.546889 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:10.549761 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:10.550412 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:10.550552 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:10.555531 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:10.555541 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:10.555545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:10.555592 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:10.559846 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:10.561015 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:10.561022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:10.561025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:10.561195 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:10.561818 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:10.561823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:10.561826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:10.562220 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:10.562371 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:10.563618 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0807 19:34:10.563867 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16503,"thread_start_us":17113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
I20260807 19:34:10.565487 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:10.526716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.129:42075)
0807 19:34:10.539050 (+ 12334us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:10.539132 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:10.541780 (+ 2648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:10.543551 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:10.543565 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:10.544354 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:10.545320 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:10.545363 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:10.547117 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:10.547124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:10.549529 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:10.549548 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:10.550906 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:10.550915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:10.551279 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:10.552201 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:10.552234 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:10.555177 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:10.560071 (+ 4894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:10.560082 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:10.560104 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:10.560843 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:10.561349 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:10.561354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:10.561359 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:10.561644 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:10.562451 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:10.562460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:10.562958 (+ 498us) client_negotiation.cc:769] Sending connection context
0807 19:34:10.563297 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 19:34:10.563930 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11940,"thread_start_us":11655,"threads_started":1}
I20260807 19:34:10.567835 22483 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:10.568217 22483 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:10.569199 22483 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:10.572156 22250 ts_manager.cc:194] Registered new tserver with Master: 8da3648960274cdaa4ec0bffe520407f (127.8.216.129:46623)
I20260807 19:34:10.574560 22250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:42075
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
W20260807 19:34:10.628623 9058 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.216.130: real 0.114s user 0.000s sys 0.024s
I20260807 19:34:10.629572 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:10 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
W20260807 19:34:11.145330 22490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:11.145895 22490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:11.146100 22490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:11.158152 22490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:11.158397 22490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:11.158644 22490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:11.174811 22490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:11.176748 22490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:11.178648 22490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:11.204941 22499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:11.205632 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:11.214910 22490 server_base.cc:1034] running on GCE node
W20260807 19:34:11.215135 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:11.216230 22490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:11.218132 22490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:11.219450 22490 hybrid_clock.cc:648] HybridClock initialized: now 1786131251219385 us; error 55 us; skew 500 ppm
I20260807 19:34:11.219883 22490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:11 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:11.225162 22490 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:11.242816 22490 webserver.cc:460] Webserver started at http://127.8.216.130:42209/ using document root <none> and password file <none>
I20260807 19:34:11.244947 22490 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:11.245364 22490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:11.246143 22490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:34:11 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:11.267525 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "41a830522a614e46b2bd66a9eb1b1f7d"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "36e9731542140023c219ecfab997d7c1"
server_key_iv: "235c56110de73c89d22eef4e40923e9f"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.268689 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "41a830522a614e46b2bd66a9eb1b1f7d"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "36e9731542140023c219ecfab997d7c1"
server_key_iv: "235c56110de73c89d22eef4e40923e9f"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.274991 22490 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260807 19:34:11.296916 22517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:11.299124 22490 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260807 19:34:11.299396 22490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "41a830522a614e46b2bd66a9eb1b1f7d"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "36e9731542140023c219ecfab997d7c1"
server_key_iv: "235c56110de73c89d22eef4e40923e9f"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.299738 22490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:11.344614 22490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:11.350464 22490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:11.350893 22490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:11.352239 22490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:11.354094 22490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:11.354194 22490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:11.354331 22490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:11.354404 22490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:11.396828 22490 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38731
I20260807 19:34:11.396888 22637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38731 every 8 connection(s)
I20260807 19:34:11.398929 22490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:11.400434 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22490
I20260807 19:34:11.400614 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:34:11.401285 9058 external_mini_cluster.cc:1428] Setting key 1cc3593f683e2a09e833c6d093bdfdeb
Aug 07 19:34:11 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
I20260807 19:34:11.453255 22640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:11.404385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:34117 (local address 127.8.216.190:36089)
0807 19:34:11.421845 (+ 17460us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:11.421850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:11.421874 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:34:11.422126 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:11.422131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:11.422233 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:11.422337 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:11.425757 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:11.426914 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:11.429593 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:11.430236 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:11.430357 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:11.435452 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:11.435460 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:11.435465 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:11.435517 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:11.448725 (+ 13208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:11.450089 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:11.450096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:11.450100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:11.450282 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:11.450908 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:11.450914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:11.450917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:11.451319 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:11.451480 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:11.452620 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0807 19:34:11.452870 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17326,"thread_start_us":17166,"threads_started":1}
I20260807 19:34:11.455700 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:11.405254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.130:34117)
0807 19:34:11.417056 (+ 11802us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:11.417135 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:11.420676 (+ 3541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:11.423742 (+ 3066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:11.423758 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:11.424507 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:11.425461 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:11.425498 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:11.427133 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:11.427139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:11.429385 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:11.429413 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:11.430735 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:11.430743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:11.431123 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:11.432020 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:11.432052 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:11.435090 (+ 3038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:11.448994 (+ 13904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:11.449008 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:11.449031 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:11.449911 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:11.450479 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:11.450485 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:11.450489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:11.450759 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:11.451580 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:11.451588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:11.452074 (+ 486us) client_negotiation.cc:769] Sending connection context
0807 19:34:11.453609 (+ 1535us) client_negotiation.cc:240] Negotiation successful
0807 19:34:11.454254 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11265,"thread_start_us":14782,"threads_started":1}
I20260807 19:34:11.457621 22638 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:11.457978 22638 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:11.458897 22638 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:11.460744 22250 ts_manager.cc:194] Registered new tserver with Master: 41a830522a614e46b2bd66a9eb1b1f7d (127.8.216.130:38731)
I20260807 19:34:11.461876 22250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:34117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:34:11.492882 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:11.579118 22483 heartbeater.cc:503] Master 127.8.216.190:36089 was elected leader, sending a full tablet report...
W20260807 19:34:11.882669 22644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:11.883093 22644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:11.883270 22644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:11.890995 22644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:11.891155 22644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:11.891328 22644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:11.901425 22644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:11.902799 22644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:11.904152 22644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:11.918016 22651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:11.918093 22650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:11.918068 22653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:11.918963 22644 server_base.cc:1034] running on GCE node
I20260807 19:34:11.919560 22644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:11.920603 22644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:11.921806 22644 hybrid_clock.cc:648] HybridClock initialized: now 1786131251921743 us; error 56 us; skew 500 ppm
I20260807 19:34:11.922137 22644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:11 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:11.925486 22644 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:11.926908 22644 webserver.cc:460] Webserver started at http://127.8.216.131:33881/ using document root <none> and password file <none>
I20260807 19:34:11.927561 22644 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:11.927662 22644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:11.927924 22644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:34:11 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:11.942600 22644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "c348a2d6d67c4385ac7de8bdd7ad5242"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "68c8b47aa50f4ebda82384c00c0cef84"
server_key_iv: "8ec87e42b47a942772d47aa0dec5f0e0"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.943514 22644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "c348a2d6d67c4385ac7de8bdd7ad5242"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "68c8b47aa50f4ebda82384c00c0cef84"
server_key_iv: "8ec87e42b47a942772d47aa0dec5f0e0"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.949280 22644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:34:11.967561 22660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:11.969251 22644 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:34:11.969450 22644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "c348a2d6d67c4385ac7de8bdd7ad5242"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "68c8b47aa50f4ebda82384c00c0cef84"
server_key_iv: "8ec87e42b47a942772d47aa0dec5f0e0"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:11.969704 22644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:11.988786 22644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:11.993100 22644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:11.993427 22644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:11.994457 22644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:11.995949 22644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:11.996042 22644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:11.996235 22644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:11.996299 22644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:12.031605 22644 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:33389
I20260807 19:34:12.031644 22790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:33389 every 8 connection(s)
I20260807 19:34:12.033269 22644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:12.043345 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22644
I20260807 19:34:12.043597 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:34:12.044181 9058 external_mini_cluster.cc:1428] Setting key 42e29e508f2564978209aeea2626c5ae
Aug 07 19:34:12 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:12.055231 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:12.036105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:50125 (local address 127.8.216.190:36089)
0807 19:34:12.036644 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:12.036649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:12.039729 (+ 3080us) server_negotiation.cc:407] Connection header received
0807 19:34:12.039797 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:12.039801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:12.039897 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:12.039985 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:12.041541 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.042305 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.043957 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.044638 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.044792 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:12.049035 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:12.049040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:12.049043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:12.049084 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:12.052365 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.053145 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.053150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.053152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.053288 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.053725 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.053729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.053731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.054018 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:12.054140 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:12.054775 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 19:34:12.054968 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":282,"threads_started":1}
I20260807 19:34:12.056329 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:12.036702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.131:50125)
0807 19:34:12.037479 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:12.037520 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:12.039437 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:12.040295 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:12.040304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:12.040746 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:12.041347 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:12.041365 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.042449 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.042453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:12.043800 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:12.043813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.044917 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.044925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:12.045233 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:12.046063 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:12.046086 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:12.048807 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:12.052506 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:12.052512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:12.052527 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:12.052996 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:12.053401 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:12.053405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:12.053407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:12.053616 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:12.054127 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:12.054133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:12.054451 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 19:34:12.054676 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 19:34:12.055085 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":294,"threads_started":1}
I20260807 19:34:12.057785 22792 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:12.058123 22792 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:12.059000 22792 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:12.060475 22250 ts_manager.cc:194] Registered new tserver with Master: c348a2d6d67c4385ac7de8bdd7ad5242 (127.8.216.131:33389)
I20260807 19:34:12.061254 22250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:50125
I20260807 19:34:12.073238 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:34:12.075126 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22335
Aug 07 19:34:12 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:34:12.093274 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:12.076370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.8.216.129:46623)
0807 19:34:12.077124 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:12.077132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:12.077161 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:34:12.077232 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:12.077238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:12.077469 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:12.077618 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:12.079144 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.080291 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.081709 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.082286 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.082398 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:12.087137 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:12.087151 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:12.087168 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:12.087211 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:12.090382 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.091103 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.091111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.091113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.091280 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.091642 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.091645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.091647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.092032 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:12.092368 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:12.092626 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:34:12.092854 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":396,"threads_started":1}
I20260807 19:34:12.094666 22478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:12.178993 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22335
I20260807 19:34:12.199903 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22490
Aug 07 19:34:12 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:34:12.225695 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:12.200986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.8.216.130:38731)
0807 19:34:12.201805 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:12.201812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:12.201839 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:34:12.201904 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:12.201911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:12.202157 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:12.202303 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:12.203562 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.204819 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.206530 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.207213 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.207319 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:12.218216 (+ 10897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:12.218238 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:12.218281 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:12.218340 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:12.222526 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.223495 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.223502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.223505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.223646 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.224098 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.224102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.224104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.224563 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:12.224818 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:12.225154 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 19:34:12.225402 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":535,"threads_started":1}
I20260807 19:34:12.227010 22633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:12.311710 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22490
I20260807 19:34:12.331292 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22644
Aug 07 19:34:12 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:34:12.348587 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:12.332348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.8.216.131:33389)
0807 19:34:12.332999 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:12.333007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:12.333037 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:34:12.333097 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:12.333103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:12.333331 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:12.333462 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:12.334751 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.335848 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.337145 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:12.337758 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:12.337858 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:12.342335 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:12.342347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:12.342363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:12.342407 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:12.345282 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.346021 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.346028 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.346031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.346170 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:12.346520 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:12.346523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:12.346525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:12.346871 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:12.347391 (+ 520us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:12.347579 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 19:34:12.347838 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":335,"threads_started":1}
I20260807 19:34:12.350076 22786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:12.444523 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22644
I20260807 19:34:12.462982 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22216
I20260807 19:34:12.464449 22281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:12.576220 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22216
I20260807 19:34:12.603232 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36089
--webserver_interface=127.8.216.190
--webserver_port=39591
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36089
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:12.948993 22804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:12.949293 22804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:12.949393 22804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:12.956851 22804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:12.956970 22804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:12.957017 22804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:12.957074 22804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:12.957122 22804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:12.967381 22804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36089
--ranger_config_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=39591
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:12.968694 22804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:12.970013 22804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:12.982102 22809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:12.982180 22812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:12.982072 22810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:12.983774 22804 server_base.cc:1034] running on GCE node
I20260807 19:34:12.984674 22804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:12.986279 22804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:12.987500 22804 hybrid_clock.cc:648] HybridClock initialized: now 1786131252987442 us; error 49 us; skew 500 ppm
I20260807 19:34:12.987896 22804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:12 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131252, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:12.991326 22804 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:12.993042 22804 webserver.cc:460] Webserver started at http://127.8.216.190:39591/ using document root <none> and password file <none>
I20260807 19:34:12.993803 22804 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:12.993980 22804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:34:13 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131252, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:13.051462 22804 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:34:13.054855 22821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:13.056485 22804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:34:13.056685 22804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "d8cbfa1e25ab4591986fd64d20b8e761"
format_stamp: "Formatted at 2026-08-07 19:34:08 on dist-test-slave-8w9v"
server_key: "e7063db7b8fe849ca88fe0ab182e66ee"
server_key_iv: "4c0dcce5edf16791254b984864fc7adf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:13.057682 22804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:13.083091 22804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:13.087560 22804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:13.087922 22804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:13.108249 22804 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36089
I20260807 19:34:13.108348 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36089 every 8 connection(s)
I20260807 19:34:13.109910 22804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:13.114012 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22804
I20260807 19:34:13.114894 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:46623
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:36089
--webserver_port=38173
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:13.117214 22875 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:34:13.120615 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Bootstrap starting.
I20260807 19:34:13.140769 22875 log.cc:824] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:13.150175 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:34:13.150745 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Bootstrap complete.
I20260807 19:34:13.160333 22875 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:13.161370 22875 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8cbfa1e25ab4591986fd64d20b8e761, State: Initialized, Role: FOLLOWER
I20260807 19:34:13.162149 22875 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:13.162425 22875 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:13.162570 22875 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:13.162772 22875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:34:13.166961 22875 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:13.167519 22875 leader_election.cc:302] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8cbfa1e25ab4591986fd64d20b8e761; no voters:
I20260807 19:34:13.168202 22875 leader_election.cc:288] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:34:13.168387 22879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:34:13.169777 22879 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [term 2 LEADER]: Becoming Leader. State: Replica: d8cbfa1e25ab4591986fd64d20b8e761, State: Running, Role: LEADER
I20260807 19:34:13.170390 22879 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } }
I20260807 19:34:13.170738 22875 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:13.173517 22881 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8cbfa1e25ab4591986fd64d20b8e761. Latest consensus state: current_term: 2 leader_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } } }
I20260807 19:34:13.173574 22880 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8cbfa1e25ab4591986fd64d20b8e761" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36089 } } }
I20260807 19:34:13.173997 22881 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:13.174046 22880 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:13.175001 22884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:13.193055 22884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:13.195014 22884 catalog_manager.cc:1259] Loaded cluster ID: 5cb84d36a42e4f75b00e827fd3226ef4
I20260807 19:34:13.195163 22884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:13.200532 22884 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:13.202476 22884 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Loaded TSK: 0
I20260807 19:34:13.205271 22884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:34:13.213179 22875 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 19:34:13.596100 22877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:13.596576 22877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:13.596750 22877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:13.608729 22877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:13.608891 22877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:13.609093 22877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:13.625542 22877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:46623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=38173
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:13.627403 22877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:13.629362 22877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:13.686401 22919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:13.690479 22918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:13.691112 22922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:13.692221 22877 server_base.cc:1034] running on GCE node
I20260807 19:34:13.693208 22877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:13.704478 22877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:13.705760 22877 hybrid_clock.cc:648] HybridClock initialized: now 1786131253705627 us; error 141 us; skew 500 ppm
I20260807 19:34:13.706187 22877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:13 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131253, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:13.709905 22877 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:13.711944 22877 webserver.cc:460] Webserver started at http://127.8.216.129:38173/ using document root <none> and password file <none>
I20260807 19:34:13.712962 22877 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:13.713136 22877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:34:13 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131253, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:13.761168 22877 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.005s
I20260807 19:34:13.766881 22931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:13.769135 22877 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:34:13.769371 22877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "8da3648960274cdaa4ec0bffe520407f"
format_stamp: "Formatted at 2026-08-07 19:34:10 on dist-test-slave-8w9v"
server_key: "fa7a97fdf2d63e340498e5d3ae1d29e8"
server_key_iv: "d9de7eb5b565c6de69566c6c08f313bf"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:13.770889 22877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:13.826510 22877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:13.831964 22877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:13.832310 22877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:13.833551 22877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:13.835361 22877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:13.835479 22877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:13.835608 22877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:13.835685 22877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:13.902112 22877 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:46623
I20260807 19:34:13.902144 23049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:46623 every 8 connection(s)
I20260807 19:34:13.904204 22877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:13.909029 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 22877
I20260807 19:34:13.909842 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:38731
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:36089
--webserver_port=42209
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:13 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131253, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:13.974990 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:13.908019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.129:49021)
0807 19:34:13.925646 (+ 17627us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:13.925723 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:13.938049 (+ 12326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:13.942593 (+ 4544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:13.942608 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:13.944168 (+ 1560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:13.945054 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:13.945090 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:13.947609 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:13.947615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:13.949722 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:13.949741 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:13.951651 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:13.951659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:13.951944 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:13.953786 (+ 1842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:13.953823 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:13.959921 (+ 6098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:13.969250 (+ 9329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:13.969263 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:13.969281 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:13.969986 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:13.970651 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:13.970657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:13.970660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:13.971084 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:13.972239 (+ 1155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:13.972248 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:13.972672 (+ 424us) client_negotiation.cc:769] Sending connection context
0807 19:34:13.972986 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 19:34:13.973617 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17266,"thread_start_us":17003,"threads_started":1}
I20260807 19:34:13.978648 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:13.907913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49021 (local address 127.8.216.190:36089)
0807 19:34:13.929393 (+ 21480us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:13.929417 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:13.929489 (+ 72us) server_negotiation.cc:407] Connection header received
0807 19:34:13.939130 (+ 9641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:13.939155 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:13.939743 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:13.941643 (+ 1900us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:13.945363 (+ 3720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:13.947355 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:13.949953 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:13.950981 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:13.951200 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:13.960392 (+ 9192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:13.960407 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:13.960433 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:13.960497 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:13.968999 (+ 8502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:13.970228 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:13.970254 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:13.970262 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:13.970492 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:13.971272 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:13.971277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:13.971281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:13.972036 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:13.976346 (+ 4310us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:13.976673 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:34:13.977196 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21105,"thread_start_us":21411,"threads_started":1}
I20260807 19:34:13.982355 23050 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:13.982745 23050 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:13.983753 23050 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
W20260807 19:34:13.987505 23050 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:36089 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:34:13.987751 23050 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:13.988055 23050 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:13.989665 23050 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:13.990061 23050 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
W20260807 19:34:13.991297 23050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:34:14.373330 23054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:14.373710 23054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:14.373871 23054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:14.382652 23054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:14.382854 23054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:14.383074 23054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:14.399509 23054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:38731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=42209
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:14.401580 23054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:14.403508 23054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:14.422235 23074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:14.423091 23072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:14.438261 23076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:14.438971 23054 server_base.cc:1034] running on GCE node
I20260807 19:34:14.439848 23054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:14.448487 23054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:14.452071 23054 hybrid_clock.cc:648] HybridClock initialized: now 1786131254451969 us; error 94 us; skew 500 ppm
I20260807 19:34:14.452677 23054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:14 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:14.457587 23054 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:14.459824 23054 webserver.cc:460] Webserver started at http://127.8.216.130:42209/ using document root <none> and password file <none>
I20260807 19:34:14.461005 23054 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:14.461213 23054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:34:14 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:14.510490 23054 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.007s sys 0.011s
Aug 07 19:34:14 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:14.524958 23084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:14.527081 23054 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.005s sys 0.008s
I20260807 19:34:14.527406 23054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "41a830522a614e46b2bd66a9eb1b1f7d"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "36e9731542140023c219ecfab997d7c1"
server_key_iv: "235c56110de73c89d22eef4e40923e9f"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:14.529286 23054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:14.559722 23054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:14.565677 23054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:14.566082 23054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:14.568850 23054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:14.570854 23054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:14.570973 23054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:14.571146 23054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:14.571259 23054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:14.627532 23054 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38731
I20260807 19:34:14.627556 23207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38731 every 8 connection(s)
I20260807 19:34:14.629865 23054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:14.638924 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23054
I20260807 19:34:14.639948 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.216.148:36625/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:33389
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:36089
--webserver_port=33881
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:14 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:14.685868 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:14.633424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:46715 (local address 127.8.216.190:36089)
0807 19:34:14.651659 (+ 18235us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:14.651666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:14.651689 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:34:14.651758 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:14.651763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:14.651855 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:14.651988 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:14.654511 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:14.655657 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:14.669870 (+ 14213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:14.670608 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:14.670756 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:14.676188 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:14.676199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:14.676205 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:14.676257 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:14.680424 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:14.681616 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:14.681625 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:14.681629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:14.681808 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:14.682470 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:14.682477 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:14.682484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:14.682958 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:14.683197 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:14.684730 (+ 1533us) server_negotiation.cc:299] Negotiation successful
0807 19:34:14.685017 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18079,"thread_start_us":17948,"threads_started":1}
I20260807 19:34:14.690122 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:14.634079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.130:46715)
0807 19:34:14.647030 (+ 12951us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:14.647102 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:14.649899 (+ 2797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:14.652584 (+ 2685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:14.652598 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:14.653324 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:14.654233 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:14.654262 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:14.667065 (+ 12803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:14.667073 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:14.669564 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:14.669591 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:14.671623 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:14.671633 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:14.672005 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:14.672982 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:14.673020 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:14.675797 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:14.680649 (+ 4852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:14.680661 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:14.680682 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:14.681424 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:14.681988 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:14.681993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:14.681996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:14.682315 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:14.683330 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:14.683339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:14.683882 (+ 543us) client_negotiation.cc:769] Sending connection context
0807 19:34:14.687554 (+ 3672us) client_negotiation.cc:240] Negotiation successful
0807 19:34:14.688523 (+ 969us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12530,"thread_start_us":16232,"threads_started":1}
I20260807 19:34:14.692605 23208 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:14.692973 23208 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:14.693883 23208 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:14.696409 22839 ts_manager.cc:194] Registered new tserver with Master: 41a830522a614e46b2bd66a9eb1b1f7d (127.8.216.130:38731)
I20260807 19:34:14.698726 22839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:46715
I20260807 19:34:14.991475 23050 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:14.991927 23050 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:14.993584 22839 ts_manager.cc:194] Registered new tserver with Master: 8da3648960274cdaa4ec0bffe520407f (127.8.216.129:46623)
I20260807 19:34:14.994760 22839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:49021
Aug 07 19:34:15 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
W20260807 19:34:15.132658 23212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:15.133037 23212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:15.133229 23212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:15.141920 23212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:15.142093 23212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:15.142292 23212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:15.158492 23212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.216.148:36625/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:33389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=33881
--tserver_master_addrs=127.8.216.190:36089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:15.160440 23212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:15.162340 23212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:15.180873 23219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:15.181830 23220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:15.188529 23212 server_base.cc:1034] running on GCE node
W20260807 19:34:15.188863 23222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:15.189917 23212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:15.191383 23212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:15.192725 23212 hybrid_clock.cc:648] HybridClock initialized: now 1786131255192651 us; error 53 us; skew 500 ppm
I20260807 19:34:15.193173 23212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:15 dist-test-slave-8w9v krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131255, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:15.197793 23212 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:15.200150 23212 webserver.cc:460] Webserver started at http://127.8.216.131:33881/ using document root <none> and password file <none>
I20260807 19:34:15.201202 23212 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:15.201426 23212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:34:15 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131255, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:15.228995 23212 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 19:34:15.232852 23230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:15.234663 23212 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:34:15.234937 23212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "c348a2d6d67c4385ac7de8bdd7ad5242"
format_stamp: "Formatted at 2026-08-07 19:34:11 on dist-test-slave-8w9v"
server_key: "68c8b47aa50f4ebda82384c00c0cef84"
server_key_iv: "8ec87e42b47a942772d47aa0dec5f0e0"
server_key_version: "kuduclusterkey@0"
I20260807 19:34:15.236523 23212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:15.278962 23212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:15.284940 23212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:15.285377 23212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:15.286675 23212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:15.288417 23212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:15.288592 23212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:15.288755 23212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:15.288885 23212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:34:15 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131254, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for HTTP/127.8.216.148@KRBTEST.COM
I20260807 19:34:15.597489 23212 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:33389
I20260807 19:34:15.597545 23348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:33389 every 8 connection(s)
I20260807 19:34:15.599212 23212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:15.601686 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23212
I20260807 19:34:15.647960 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.603041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.8.216.190:36089)
0807 19:34:15.621131 (+ 18090us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.621141 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.621171 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:34:15.621287 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.621295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.621405 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.621567 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:15.623672 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.624952 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.634146 (+ 9194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.634940 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.635112 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.638951 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:15.638958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:15.638963 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:15.639012 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:15.643122 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.644156 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.644162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.644165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.644347 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.644891 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.644896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.644900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.645299 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:15.647171 (+ 1872us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.647326 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.647572 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17903,"thread_start_us":18713,"threads_started":1}
Aug 07 19:34:15 dist-test-slave-8w9v krb5kdc[21741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131255, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:15.658022 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.606423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:58661 (local address 127.8.216.190:36089)
0807 19:34:15.636048 (+ 29625us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.636054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.636076 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:34:15.637265 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.637271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.637359 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.637468 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:15.639940 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.641262 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.644068 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.644784 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.644882 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.650538 (+ 5656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:15.650546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:15.650551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:15.650594 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:15.654026 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.655062 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.655068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.655071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.655216 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.655832 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.655837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.655840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.656254 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:15.656374 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.657382 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.657612 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29460,"thread_start_us":31186,"threads_started":1}
I20260807 19:34:15.660429 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.605390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36089 (local address 127.8.216.131:58661)
0807 19:34:15.632728 (+ 27338us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.632808 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.636804 (+ 3996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.638102 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.638116 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.638796 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:15.639692 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.639719 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.641480 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.641486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.643904 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.643920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.645320 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.645328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.645615 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.647316 (+ 1701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:15.647353 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:15.650253 (+ 2900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:15.654205 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:15.654215 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:15.654234 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:15.654908 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:15.655392 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:15.655398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:15.655401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:15.655679 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:15.656481 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:15.656490 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:15.656941 (+ 451us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.658312 (+ 1371us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.658920 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26966,"thread_start_us":29096,"threads_started":1}
I20260807 19:34:15.662740 23349 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36089
I20260807 19:34:15.663086 23349 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:15.664026 23349 heartbeater.cc:511] Master 127.8.216.190:36089 requested a full tablet report, sending...
I20260807 19:34:15.665975 22839 ts_manager.cc:194] Registered new tserver with Master: c348a2d6d67c4385ac7de8bdd7ad5242 (127.8.216.131:33389)
I20260807 19:34:15.667184 22839 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:58661
I20260807 19:34:15.670674 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:34:15.692965 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.677638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.8.216.190:36089)
0807 19:34:15.677855 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.677861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.678468 (+ 607us) server_negotiation.cc:407] Connection header received
0807 19:34:15.678620 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.678626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.678710 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.678831 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:15.680456 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.681576 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.682964 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.683612 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.683812 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.686443 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:15.686450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:15.686455 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:15.686498 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:15.690035 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.690792 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.690797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.690801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.690972 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:15.691366 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:15.691371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:15.691374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:15.691761 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:15.692007 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.692369 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.692614 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:34:15.703759 23208 heartbeater.cc:503] Master 127.8.216.190:36089 was elected leader, sending a full tablet report...
I20260807 19:34:15.698781 22839 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:34:15.710258 22839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:34:15.769116 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.726650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:33389 (local address 127.0.0.1:35522)
0807 19:34:15.758058 (+ 31408us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.758076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.758189 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.759679 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.759684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.759710 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:15.760039 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.760049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.762508 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.762513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.765669 (+ 3156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.765690 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.767968 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.767975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.768218 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.768277 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.768419 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.768632 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31293,"mutex_wait_us":66,"thread_start_us":37492,"threads_started":1}
I20260807 19:34:15.770370 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.727303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.8.216.131:33389)
0807 19:34:15.755730 (+ 28427us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.755744 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.758307 (+ 2563us) server_negotiation.cc:407] Connection header received
0807 19:34:15.758417 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.758425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.758624 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.758827 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:15.760302 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.762346 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.765887 (+ 3541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.767413 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.767548 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.767793 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.769811 (+ 2018us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.770009 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28144,"thread_start_us":28449,"threads_started":1}
I20260807 19:34:15.771330 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.726643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.8.216.129:46623)
0807 19:34:15.756738 (+ 30095us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.756748 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.757677 (+ 929us) server_negotiation.cc:407] Connection header received
0807 19:34:15.757822 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.757831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.758117 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.758325 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:15.759457 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.761713 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.765887 (+ 4174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.768686 (+ 2799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.768818 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.769061 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.770803 (+ 1742us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.770990 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29892,"thread_start_us":30124,"threads_started":1}
I20260807 19:34:15.772650 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.726811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.8.216.130:38731)
0807 19:34:15.762368 (+ 35557us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.762374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.762395 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:34:15.762538 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.762549 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.762797 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.763008 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:15.764975 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.766591 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.769008 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.770398 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.770567 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.770882 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.772147 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.772322 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35373,"thread_start_us":35208,"threads_started":1}
I20260807 19:34:15.772837 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.726152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43434)
0807 19:34:15.756822 (+ 30670us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.756888 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.757217 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.758644 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.758651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.758681 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:15.759167 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.759181 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.762782 (+ 3601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.762788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.765690 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.765700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.769352 (+ 3652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.769360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.769532 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.769563 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.769651 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.769812 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30287,"mutex_wait_us":51,"thread_start_us":32913,"threads_started":1}
I20260807 19:34:15.773890 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.726270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:38731 (local address 127.0.0.1:40712)
0807 19:34:15.757387 (+ 31117us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.757407 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.757567 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.763272 (+ 5705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.763276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.763303 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:15.764726 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.764738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.766798 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.766803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.768813 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.768829 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.771105 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.771112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.771281 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.771311 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.771403 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.771880 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30978,"mutex_wait_us":951,"thread_start_us":31530,"threads_started":1}
I20260807 19:34:15.774928 23282 tablet_service.cc:1467] Processing CreateTablet for tablet bdcc153ec3814281800e50f4cb2fc423 (DEFAULT_TABLE table=test-table [id=f51775199ea34227b4d888373bd26339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:15.776718 23282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcc153ec3814281800e50f4cb2fc423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:15.776645 22979 tablet_service.cc:1467] Processing CreateTablet for tablet bdcc153ec3814281800e50f4cb2fc423 (DEFAULT_TABLE table=test-table [id=f51775199ea34227b4d888373bd26339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:15.778124 22979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcc153ec3814281800e50f4cb2fc423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:15.777853 23134 tablet_service.cc:1467] Processing CreateTablet for tablet bdcc153ec3814281800e50f4cb2fc423 (DEFAULT_TABLE table=test-table [id=f51775199ea34227b4d888373bd26339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:15.779270 23134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdcc153ec3814281800e50f4cb2fc423. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:15.799314 23377 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Bootstrap starting.
I20260807 19:34:15.807137 23377 tablet_bootstrap.cc:654] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:15.808966 23378 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Bootstrap starting.
I20260807 19:34:15.811380 23379 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Bootstrap starting.
I20260807 19:34:15.813046 23378 tablet_bootstrap.cc:654] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:15.814436 23378 log.cc:824] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:15.815199 23379 tablet_bootstrap.cc:654] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:15.816864 23379 log.cc:824] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:15.818372 23377 log.cc:824] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:15.823702 23379 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: No bootstrap required, opened a new log
I20260807 19:34:15.824034 23379 ts_tablet_manager.cc:1397] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20260807 19:34:15.825443 23378 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: No bootstrap required, opened a new log
I20260807 19:34:15.825748 23378 ts_tablet_manager.cc:1397] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260807 19:34:15.827858 23377 tablet_bootstrap.cc:492] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: No bootstrap required, opened a new log
I20260807 19:34:15.828226 23377 ts_tablet_manager.cc:1397] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.004s
I20260807 19:34:15.836424 23379 raft_consensus.cc:348] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.836589 23378 raft_consensus.cc:348] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.836953 23379 raft_consensus.cc:374] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:15.837029 23378 raft_consensus.cc:374] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:15.837047 23379 raft_consensus.cc:729] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da3648960274cdaa4ec0bffe520407f, State: Initialized, Role: FOLLOWER
I20260807 19:34:15.837141 23378 raft_consensus.cc:729] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a830522a614e46b2bd66a9eb1b1f7d, State: Initialized, Role: FOLLOWER
I20260807 19:34:15.837783 23379 consensus_queue.cc:260] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.837855 23378 consensus_queue.cc:260] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.840916 23377 raft_consensus.cc:348] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.841574 23377 raft_consensus.cc:374] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:15.841728 23377 raft_consensus.cc:729] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c348a2d6d67c4385ac7de8bdd7ad5242, State: Initialized, Role: FOLLOWER
I20260807 19:34:15.842581 23377 consensus_queue.cc:260] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.843289 23378 ts_tablet_manager.cc:1428] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260807 19:34:15.853135 23050 heartbeater.cc:503] Master 127.8.216.190:36089 was elected leader, sending a full tablet report...
I20260807 19:34:15.853822 23379 ts_tablet_manager.cc:1428] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Time spent starting tablet: real 0.030s user 0.016s sys 0.009s
I20260807 19:34:15.855398 23349 heartbeater.cc:503] Master 127.8.216.190:36089 was elected leader, sending a full tablet report...
I20260807 19:34:15.856318 23377 ts_tablet_manager.cc:1428] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Time spent starting tablet: real 0.028s user 0.017s sys 0.009s
I20260807 19:34:15.856868 23386 raft_consensus.cc:482] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:15.857234 23386 raft_consensus.cc:504] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.860917 23386 leader_election.cc:288] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8da3648960274cdaa4ec0bffe520407f (127.8.216.129:46623), c348a2d6d67c4385ac7de8bdd7ad5242 (127.8.216.131:33389)
I20260807 19:34:15.868021 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.859999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:60819 (local address 127.8.216.131:33389)
0807 19:34:15.860229 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.860238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.862153 (+ 1915us) server_negotiation.cc:407] Connection header received
0807 19:34:15.862325 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.862333 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.862436 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.862535 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:15.863378 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.864433 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.866203 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.867000 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.867202 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.867289 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.867620 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.867752 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:34:15.868085 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.859833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:33389 (local address 127.8.216.130:60819)
0807 19:34:15.861982 (+ 2149us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.862004 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.862182 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.862667 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.862674 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.862701 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:15.863180 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.863200 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.864585 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.864589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.866028 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.866039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.867260 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.867266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.867444 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.867471 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.867591 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.867732 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1950,"thread_start_us":2577,"threads_started":1}
I20260807 19:34:15.869545 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcc153ec3814281800e50f4cb2fc423" candidate_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" is_pre_election: true
I20260807 19:34:15.870141 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.860447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:57627 (local address 127.8.216.129:46623)
0807 19:34:15.860699 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.860707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.862947 (+ 2240us) server_negotiation.cc:407] Connection header received
0807 19:34:15.863219 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.863243 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.863318 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.863379 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:15.864195 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.865301 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.867818 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.869099 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.869272 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.869352 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.869719 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.869870 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:34:15.870150 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.859910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.8.216.130:57627)
0807 19:34:15.862831 (+ 2921us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:15.862851 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:15.863059 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:15.863573 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:15.863578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:15.863603 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:15.863980 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.863992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.865498 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.865503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.867622 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:15.867639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.869313 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.869319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:15.869506 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.869552 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 19:34:15.869662 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:34:15.869828 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2706,"thread_start_us":1645,"threads_started":1}
I20260807 19:34:15.870187 23303 raft_consensus.cc:2454] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41a830522a614e46b2bd66a9eb1b1f7d in term 0.
I20260807 19:34:15.871014 23089 leader_election.cc:302] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a830522a614e46b2bd66a9eb1b1f7d, c348a2d6d67c4385ac7de8bdd7ad5242; no voters:
I20260807 19:34:15.871503 23386 raft_consensus.cc:2781] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:34:15.871424 22999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcc153ec3814281800e50f4cb2fc423" candidate_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3648960274cdaa4ec0bffe520407f" is_pre_election: true
I20260807 19:34:15.871670 23386 raft_consensus.cc:482] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:34:15.871763 23386 raft_consensus.cc:3037] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:15.872059 22999 raft_consensus.cc:2454] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41a830522a614e46b2bd66a9eb1b1f7d in term 0.
W20260807 19:34:15.872951 23350 tablet.cc:2369] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:15.875144 23386 raft_consensus.cc:504] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.875878 23386 leader_election.cc:288] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [CANDIDATE]: Term 1 election: Requested vote from peers 8da3648960274cdaa4ec0bffe520407f (127.8.216.129:46623), c348a2d6d67c4385ac7de8bdd7ad5242 (127.8.216.131:33389)
I20260807 19:34:15.876461 23303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcc153ec3814281800e50f4cb2fc423" candidate_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242"
I20260807 19:34:15.876688 23303 raft_consensus.cc:3037] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:15.879988 23303 raft_consensus.cc:2454] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41a830522a614e46b2bd66a9eb1b1f7d in term 1.
I20260807 19:34:15.880466 23089 leader_election.cc:302] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41a830522a614e46b2bd66a9eb1b1f7d, c348a2d6d67c4385ac7de8bdd7ad5242; no voters:
I20260807 19:34:15.880457 22999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdcc153ec3814281800e50f4cb2fc423" candidate_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8da3648960274cdaa4ec0bffe520407f"
I20260807 19:34:15.880688 22999 raft_consensus.cc:3037] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:15.880795 23386 raft_consensus.cc:2781] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:15.881366 23386 raft_consensus.cc:686] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 1 LEADER]: Becoming Leader. State: Replica: 41a830522a614e46b2bd66a9eb1b1f7d, State: Running, Role: LEADER
I20260807 19:34:15.881940 23386 consensus_queue.cc:237] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } }
I20260807 19:34:15.883697 22999 raft_consensus.cc:2454] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41a830522a614e46b2bd66a9eb1b1f7d in term 1.
I20260807 19:34:15.887686 22838 catalog_manager.cc:5495] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d reported cstate change: term changed from 0 to 1, leader changed from <none> to 41a830522a614e46b2bd66a9eb1b1f7d (127.8.216.130). New cstate: current_term: 1 leader_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41a830522a614e46b2bd66a9eb1b1f7d" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 } health_report { overall_health: UNKNOWN } } }
W20260807 19:34:15.907560 23209 tablet.cc:2369] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:15.931587 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.925745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.8.216.130:38731)
0807 19:34:15.925980 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.925989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.926056 (+ 67us) server_negotiation.cc:407] Connection header received
0807 19:34:15.926330 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.926333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.926415 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.926583 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:34:15.926992 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.927769 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.929296 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.929713 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.929864 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.930415 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:34:15.930968 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:34:15.931149 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:15.931206 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 19:34:15.931296 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:34:15.940519 22999 raft_consensus.cc:1261] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 1 FOLLOWER]: Refusing update from remote peer 41a830522a614e46b2bd66a9eb1b1f7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:15.940519 23303 raft_consensus.cc:1261] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 1 FOLLOWER]: Refusing update from remote peer 41a830522a614e46b2bd66a9eb1b1f7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:15.941738 23391 consensus_queue.cc:1035] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8da3648960274cdaa4ec0bffe520407f" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:15.942175 23386 consensus_queue.cc:1035] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c348a2d6d67c4385ac7de8bdd7ad5242" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 33389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:34:15.946069 23051 tablet.cc:2369] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:15.961387 23397 mvcc.cc:205] Tried to move back new op lower bound from 7315993624320450560 to 7315993624097193984. Current Snapshot: MvccSnapshot[applied={T|T < 7315993624320450560 or (T in {7315993624320450560})}]
I20260807 19:34:15.968422 22839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:15.968703 22839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:15.972733 22839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8cbfa1e25ab4591986fd64d20b8e761: Sending DeleteTablet for 3 replicas of tablet bdcc153ec3814281800e50f4cb2fc423
I20260807 19:34:15.974002 22979 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcc153ec3814281800e50f4cb2fc423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:15 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:43434
I20260807 19:34:15.973994 23134 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcc153ec3814281800e50f4cb2fc423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:15 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:40712
I20260807 19:34:15.974125 23282 tablet_service.cc:1514] Processing DeleteTablet for tablet bdcc153ec3814281800e50f4cb2fc423 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:15 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:35522
I20260807 19:34:15.984946 23406 tablet_replica.cc:330] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: stopping tablet replica
I20260807 19:34:15.985399 23406 raft_consensus.cc:2229] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:34:15.986023 23406 raft_consensus.cc:2258] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:15.986490 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22877
I20260807 19:34:15.988998 23406 ts_tablet_manager.cc:1905] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:15.991781 23407 tablet_replica.cc:330] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: stopping tablet replica
I20260807 19:34:15.992295 23407 raft_consensus.cc:2229] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:15.992837 23407 raft_consensus.cc:2258] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:15.995072 23405 tablet_replica.cc:330] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: stopping tablet replica
I20260807 19:34:15.995231 23407 ts_tablet_manager.cc:1905] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:15.995502 23405 raft_consensus.cc:2229] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:15.996031 23405 raft_consensus.cc:2258] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:15.998070 23406 ts_tablet_manager.cc:1918] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:15.998311 23406 log.cc:1196] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/bdcc153ec3814281800e50f4cb2fc423
I20260807 19:34:15.998996 23406 ts_tablet_manager.cc:1939] T bdcc153ec3814281800e50f4cb2fc423 P 41a830522a614e46b2bd66a9eb1b1f7d: Deleting consensus metadata
I20260807 19:34:16.001403 22822 catalog_manager.cc:4841] TS 41a830522a614e46b2bd66a9eb1b1f7d (127.8.216.130:38731): tablet bdcc153ec3814281800e50f4cb2fc423 (table test-table [id=f51775199ea34227b4d888373bd26339]) successfully deleted
I20260807 19:34:16.002224 23405 ts_tablet_manager.cc:1905] T bdcc153ec3814281800e50f4cb2fc423 P 8da3648960274cdaa4ec0bffe520407f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:16.002820 23407 ts_tablet_manager.cc:1918] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:16.003012 23407 log.cc:1196] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/bdcc153ec3814281800e50f4cb2fc423
I20260807 19:34:16.003656 23407 ts_tablet_manager.cc:1939] T bdcc153ec3814281800e50f4cb2fc423 P c348a2d6d67c4385ac7de8bdd7ad5242: Deleting consensus metadata
I20260807 19:34:16.005396 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:15.987696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43438 (local address 127.8.216.129:46623)
0807 19:34:15.987905 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:15.987911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:15.987934 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:34:15.988018 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:15.988023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:15.988149 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:15.988270 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:15.990390 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.991517 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.993398 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:15.994078 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:15.994216 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:15.997657 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:15.997672 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:15.997691 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:15.997755 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:16.001697 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.002650 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.002659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.002662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.002856 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.003410 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.003421 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.003424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.004017 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:16.004403 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:16.004718 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 19:34:16.004979 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:34:16.005613 22825 catalog_manager.cc:4841] TS c348a2d6d67c4385ac7de8bdd7ad5242 (127.8.216.131:33389): tablet bdcc153ec3814281800e50f4cb2fc423 (table test-table [id=f51775199ea34227b4d888373bd26339]) successfully deleted
I20260807 19:34:16.006469 23045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:16.100281 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22877
I20260807 19:34:16.129534 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23054
W20260807 19:34:16.129547 22824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:46623 (error 108)
I20260807 19:34:16.133826 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.133246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43454)
0807 19:34:16.133472 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.133627 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 19:34:16.134148 22824 catalog_manager.cc:4584] TS 8da3648960274cdaa4ec0bffe520407f (127.8.216.129:46623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bdcc153ec3814281800e50f4cb2fc423: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
I20260807 19:34:16.146952 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.130479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.8.216.130:38731)
0807 19:34:16.130679 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:16.130686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:16.131112 (+ 426us) server_negotiation.cc:407] Connection header received
0807 19:34:16.131193 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:16.131201 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:16.131328 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:16.131552 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:16.133012 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:16.134073 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:16.135689 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:16.136485 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:16.136635 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:16.139356 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:16.139381 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:16.139402 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:16.139463 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:16.143773 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.144626 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.144635 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.144639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.144844 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.145231 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.145236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.145239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.145729 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:16.146107 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:16.146268 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 19:34:16.146527 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:34:16.147779 23203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:16.188093 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.187594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43460)
0807 19:34:16.187837 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.187942 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 19:34:16.189226 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.188854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43474)
0807 19:34:16.189023 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.189100 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:34:16.249884 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.249289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43488)
0807 19:34:16.249557 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.249660 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 19:34:16.251050 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.250669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43500)
0807 19:34:16.250846 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.250928 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:34:16.265235 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23054
I20260807 19:34:16.287673 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23212
I20260807 19:34:16.302690 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.288517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35538 (local address 127.8.216.131:33389)
0807 19:34:16.288714 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:16.288719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:16.288739 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:34:16.288887 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:16.288892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:16.288960 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:16.289046 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:16.290481 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:16.291349 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:16.292867 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:16.293305 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:16.293408 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:16.296540 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:16.296557 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:16.296575 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:16.296620 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:16.299572 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.300601 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.300609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.300611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.300749 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:16.301094 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:16.301097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:16.301099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:16.301489 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:16.301728 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:16.302068 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 19:34:16.302300 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:34:16.303510 23344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:16.331797 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.331253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43506)
0807 19:34:16.331514 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.331619 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 19:34:16.333072 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.332640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43522)
0807 19:34:16.332872 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.332949 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260807 19:34:16.396039 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23212
I20260807 19:34:16.416643 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 22804
I20260807 19:34:16.417853 22870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:16.555914 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:16.555287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46623 (local address 127.0.0.1:43534)
0807 19:34:16.555496 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:16.555630 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:34:16.556329 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 22804
I20260807 19:34:16.588279 9058 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:34:16.934777 9058 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:34:16.935045 9058 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:34:17.291111 9058 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:34:17.291 UTC [21772] LOG: received smart shutdown request
2026-08-07 19:34:17.295 UTC [21772] LOG: background worker "logical replication launcher" (PID 21782) exited with exit code 1
2026-08-07 19:34:17.295 UTC [21777] LOG: shutting down
2026-08-07 19:34:17.492 UTC [21772] LOG: database system is shut down
2026-08-07T19:34:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56724 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:34:17 dist-test-slave-8w9v krb5kdc[23422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:34:17 dist-test-slave-8w9v krb5kdc[23422](info): set up 2 sockets
Aug 07 19:34:17 dist-test-slave-8w9v krb5kdc[23422](info): commencing operation
krb5kdc: starting...
W20260807 19:34:18.799921 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:34:18 dist-test-slave-8w9v krb5kdc[23422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:34:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:34:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:34:19.112203 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:43425
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:19.475589 23438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:19.475910 23438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:19.476109 23438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:19.483765 23438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:19.483894 23438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:19.483995 23438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:19.484083 23438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:19.484205 23438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:19.494702 23438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:19.496284 23438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:19.497731 23438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:19.510396 23444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:19.510311 23445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:19.510430 23447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:19.511305 23438 server_base.cc:1034] running on GCE node
I20260807 19:34:19.512264 23438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:19.513772 23438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:19.515021 23438 hybrid_clock.cc:648] HybridClock initialized: now 1786131259514979 us; error 51 us; skew 500 ppm
I20260807 19:34:19.515389 23438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:19 dist-test-slave-8w9v krb5kdc[23422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:19.518848 23438 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:19.520591 23438 webserver.cc:460] Webserver started at http://127.8.216.190:45857/ using document root <none> and password file <none>
I20260807 19:34:19.521299 23438 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:19.521405 23438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:19.521685 23438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:19.524958 23438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "21b66e801b7d404c910319b7c7cae22e"
format_stamp: "Formatted at 2026-08-07 19:34:19 on dist-test-slave-8w9v"
server_key: "c2555bc00ad18ab993342b3c251fb1ce"
server_key_iv: "ae9b2e19018567428269d5af215a2c95"
server_key_version: "encryptionkey@0"
I20260807 19:34:19.525770 23438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "21b66e801b7d404c910319b7c7cae22e"
format_stamp: "Formatted at 2026-08-07 19:34:19 on dist-test-slave-8w9v"
server_key: "c2555bc00ad18ab993342b3c251fb1ce"
server_key_iv: "ae9b2e19018567428269d5af215a2c95"
server_key_version: "encryptionkey@0"
I20260807 19:34:19.531692 23438 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:34:19.535404 23454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:19.536916 23438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:34:19.537108 23438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "21b66e801b7d404c910319b7c7cae22e"
format_stamp: "Formatted at 2026-08-07 19:34:19 on dist-test-slave-8w9v"
server_key: "c2555bc00ad18ab993342b3c251fb1ce"
server_key_iv: "ae9b2e19018567428269d5af215a2c95"
server_key_version: "encryptionkey@0"
I20260807 19:34:19.537349 23438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:19.558334 23438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:19.562739 23438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:19.563077 23438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:19.583673 23438 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:43425
I20260807 19:34:19.583707 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:43425 every 8 connection(s)
I20260807 19:34:19.585275 23438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:19.589516 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:19.592895 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23438
I20260807 19:34:19.593168 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:34:19.594007 9058 external_mini_cluster.cc:1428] Setting key e87f71ea20fba093b91e01160f359be4
I20260807 19:34:19.599330 23508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:34:19 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.189@KRBTEST.COM" created.
I20260807 19:34:19.628530 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.603782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:43425 (local address 127.0.0.1:44366)
0807 19:34:19.605721 (+ 1939us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.605765 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:19.608444 (+ 2679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:19.609436 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:19.609443 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:19.610412 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:19.610967 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:19.610998 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:19.612826 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:19.612835 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:19.614902 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:19.614924 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:19.616007 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:19.616017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:19.616287 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:19.617099 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:19.617136 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:19.620510 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:19.623998 (+ 3488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:19.624011 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:19.624025 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:19.624665 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:19.625106 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:19.625109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:19.625112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:19.625309 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:19.626195 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:19.626202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:19.626421 (+ 219us) client_negotiation.cc:769] Sending connection context
0807 19:34:19.626638 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 19:34:19.627073 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"thread_start_us":1462,"threads_started":1}
I20260807 19:34:19.628561 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.603782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.8.216.190:43425)
0807 19:34:19.605174 (+ 1392us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:19.605195 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:19.605951 (+ 756us) server_negotiation.cc:407] Connection header received
0807 19:34:19.608890 (+ 2939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:19.608896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:19.609159 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:19.609342 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:19.611211 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:19.612643 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:19.615096 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:19.615712 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:19.615854 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:19.620779 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:19.620788 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:19.620802 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:19.620841 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:19.623809 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:19.624817 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:19.624825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:19.624827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:19.624993 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:19.625469 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:19.625477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:19.625484 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:19.626067 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:19.626236 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:19.626844 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 19:34:19.627468 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":162,"server-negotiator.queue_time_us":1078,"spinlock_wait_cycles":18304,"thread_start_us":747,"threads_started":1}
I20260807 19:34:19.631806 23508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:19.633018 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.632593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60898)
0807 19:34:19.632780 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.632900 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:34:19.633258 23458 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
I20260807 19:34:19.634931 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.634546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60904)
0807 19:34:19.634725 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.634805 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:34:19.635138 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:38079: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
I20260807 19:34:19.663383 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:38079
--webserver_interface=127.8.216.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:19.684520 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } attempt: 1
I20260807 19:34:19.686040 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.685397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60906)
0807 19:34:19.685671 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.685820 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 19:34:19.687314 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.686820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60916)
0807 19:34:19.687009 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.687111 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 19:34:19.687582 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:38079: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
I20260807 19:34:19.755883 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } attempt: 2
I20260807 19:34:19.757285 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.756682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60918)
0807 19:34:19.756968 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.757092 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:34:19.758641 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.758161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60920)
0807 19:34:19.758336 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.758460 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:34:19.758906 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:38079: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
I20260807 19:34:19.850258 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } attempt: 3
I20260807 19:34:19.851567 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.851001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60936)
0807 19:34:19.851256 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.851377 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 19:34:19.852839 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.852359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60950)
0807 19:34:19.852557 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.852668 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:34:19.853097 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:38079: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
I20260807 19:34:19.996379 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } attempt: 4
I20260807 19:34:19.997979 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.997276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60952)
0807 19:34:19.997677 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.997809 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233}
I20260807 19:34:19.999397 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:19.998924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60962)
0807 19:34:19.999117 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:19.999237 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 19:34:19.999660 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:38079: Network error: Client connection negotiation failed: client connection to 127.8.216.189:38079: connect: Connection refused (error 111)
W20260807 19:34:20.027938 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:20.028457 23515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:20.028656 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:20.036545 23515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:20.036717 23515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:20.036839 23515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:20.036942 23515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:20.037030 23515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:20.048286 23515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:38079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.216.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:20.049853 23515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:20.051265 23515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:20.064568 23524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:20.064693 23525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:20.065122 23527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:20.065759 23515 server_base.cc:1034] running on GCE node
I20260807 19:34:20.066704 23515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:20.068409 23515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:20.069656 23515 hybrid_clock.cc:648] HybridClock initialized: now 1786131260069592 us; error 63 us; skew 500 ppm
I20260807 19:34:20.070050 23515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:20.073828 23515 init.cc:377] Logged in from keytab as kudu/127.8.216.189@KRBTEST.COM (short username kudu)
I20260807 19:34:20.075582 23515 webserver.cc:460] Webserver started at http://127.8.216.189:33411/ using document root <none> and password file <none>
I20260807 19:34:20.076452 23515 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:20.076614 23515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:20.076946 23515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:20.080363 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data/instance:
uuid: "1df932e760234a6dbaae37c180acc3bc"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "c3a367fb6b5ea81b87ccd7e475665753"
server_key_iv: "bf4e060ea584ef5471b59af9d3f28ad4"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.081298 23515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal/instance:
uuid: "1df932e760234a6dbaae37c180acc3bc"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "c3a367fb6b5ea81b87ccd7e475665753"
server_key_iv: "bf4e060ea584ef5471b59af9d3f28ad4"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.087100 23515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:34:20.090946 23534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:20.092597 23515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:34:20.092829 23515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "1df932e760234a6dbaae37c180acc3bc"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "c3a367fb6b5ea81b87ccd7e475665753"
server_key_iv: "bf4e060ea584ef5471b59af9d3f28ad4"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.093062 23515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:20.114706 23515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:20.119563 23515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:20.119886 23515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:20.140430 23515 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.189:38079
I20260807 19:34:20.140458 23589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.189:38079 every 8 connection(s)
I20260807 19:34:20.142261 23515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
I20260807 19:34:20.144098 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23515
I20260807 19:34:20.144357 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-1/wal/instance
I20260807 19:34:20.144848 9058 external_mini_cluster.cc:1428] Setting key e9894dd141748231ade6fdce5f4c7d79
I20260807 19:34:20.146410 23590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:20.153219 23590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.188@KRBTEST.COM" created.
I20260807 19:34:20.174922 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.155984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.8.216.190:43425)
0807 19:34:20.156537 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.156542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.157515 (+ 973us) server_negotiation.cc:407] Connection header received
0807 19:34:20.159466 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.159472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.159577 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.159704 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.161322 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.162146 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.163730 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.164243 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.164387 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.168377 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.168382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.168385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.168425 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.171493 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.172430 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.172435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.172443 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.172575 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.173078 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.173082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.173084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.173368 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.173563 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.174419 (+ 856us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.174630 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":284,"threads_started":1}
I20260807 19:34:20.176455 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.156548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:43425 (local address 127.0.0.1:44370)
0807 19:34:20.157338 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.157382 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.159171 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.159975 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.159983 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.160575 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.161142 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.161159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.162324 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.162328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.163574 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.163588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.164486 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.164495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.164751 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.165510 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.165534 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.168064 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.171673 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.171684 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.171703 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.172285 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.172703 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.172708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.172712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.172935 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.173521 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.173529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.173932 (+ 403us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.174230 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.174796 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":349,"threads_started":1}
I20260807 19:34:20.178404 23590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:34:20.195827 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.179205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60972)
0807 19:34:20.179443 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.179465 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.179585 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.180470 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.180477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.181017 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.181252 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.181268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.182650 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.182658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.184194 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.184215 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.185077 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.185086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.185355 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.186147 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.186166 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.188562 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.193252 (+ 4690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.193256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.193259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.193648 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.194092 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.194096 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.194098 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.194263 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.195085 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.195089 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.195217 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.195330 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.195554 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:34:20.196466 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.179326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60972 (local address 127.8.216.189:38079)
0807 19:34:20.179972 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.179978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.180004 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:34:20.180058 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.180065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.180302 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.180430 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.181424 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.182495 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.184339 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.184908 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.185008 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.188818 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.188839 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.188859 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.188903 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.193086 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.193796 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.193808 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.193814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.193969 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.194377 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.194384 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.194390 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.194951 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.195185 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.195517 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.195818 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":338,"threads_started":1}
I20260807 19:34:20.199258 23590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:20.200533 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.199997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60754)
0807 19:34:20.200241 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.200369 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 19:34:20.200796 23540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.202530 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.202154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60766)
0807 19:34:20.202341 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.202420 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:34:20.202754 23590 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
I20260807 19:34:20.213363 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:35373
--webserver_interface=127.8.216.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:20.252185 23590 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 1
I20260807 19:34:20.253584 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.252997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60780)
0807 19:34:20.253234 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.253390 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:34:20.254828 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.254360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60782)
0807 19:34:20.254566 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.254675 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:34:20.255072 23590 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.299008 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } attempt: 5
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:34:20.314405 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.299925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.8.216.189:38079)
0807 19:34:20.300170 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.300177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.300207 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:34:20.300418 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.300422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.300522 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.300615 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.301715 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.302398 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.304413 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.304853 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.304953 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.308945 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.308952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.308955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.308992 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.311679 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.312573 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.312577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.312580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.312701 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.313147 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.313150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.313152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.313430 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.313552 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.314000 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.314182 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:34:20.314584 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.299804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60982)
0807 19:34:20.300055 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.300081 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.300279 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.300706 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.300713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.301258 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.301546 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.301563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.302574 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.302582 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.304264 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.304283 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.305021 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.305027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.305224 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.305998 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.306022 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.308715 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.311837 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.311843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.311847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.312439 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.312825 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.312831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.312837 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.313031 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.313562 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.313567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.313749 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.313909 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.314236 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:34:20.315961 23508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:20.317269 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.316775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60792)
0807 19:34:20.316984 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.317089 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:34:20.318272 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.317894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60800)
0807 19:34:20.318043 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.318142 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:34:20.318483 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.323308 23590 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 2
I20260807 19:34:20.324486 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.324002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60804)
0807 19:34:20.324246 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.324333 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:34:20.325440 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.325110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60816)
0807 19:34:20.325258 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.325330 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:34:20.325644 23590 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.369767 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 1
I20260807 19:34:20.371178 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.370556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60824)
0807 19:34:20.370821 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.370957 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:34:20.372344 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.371906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60826)
0807 19:34:20.372077 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.372230 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 19:34:20.372610 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.416882 23590 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 3
I20260807 19:34:20.418263 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.417679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60834)
0807 19:34:20.417984 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.418082 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 19:34:20.419431 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.418995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60838)
0807 19:34:20.419243 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.419321 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 19:34:20.419648 23590 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.440882 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 2
I20260807 19:34:20.442196 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.441660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60846)
0807 19:34:20.441888 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.442020 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:34:20.443356 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.442960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60852)
0807 19:34:20.443142 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.443234 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 19:34:20.443575 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.549824 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 3
I20260807 19:34:20.551103 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.550610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60862)
0807 19:34:20.550831 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.550921 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:34:20.552289 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.551853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60866)
0807 19:34:20.552040 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.552159 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:34:20.552490 23508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
I20260807 19:34:20.562894 23590 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 4
I20260807 19:34:20.564113 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.563594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60876)
0807 19:34:20.563840 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.563929 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:34:20.565176 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.564817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60884)
0807 19:34:20.564980 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.565071 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:34:20.565378 23590 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:35373: Network error: Client connection negotiation failed: client connection to 127.8.216.188:35373: connect: Connection refused (error 111)
W20260807 19:34:20.586095 23599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:20.586503 23599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:20.586635 23599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:20.597726 23599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:20.597858 23599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:20.597909 23599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:20.597962 23599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:20.597999 23599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:20.608616 23599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:43425,127.8.216.189:38079,127.8.216.188:35373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:35373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.216.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:20.610102 23599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:20.611589 23599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:20.625128 23604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:20.625517 23605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:20.625591 23607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:20.626505 23599 server_base.cc:1034] running on GCE node
I20260807 19:34:20.627383 23599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:20.629050 23599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:20.630297 23599 hybrid_clock.cc:648] HybridClock initialized: now 1786131260630249 us; error 44 us; skew 500 ppm
I20260807 19:34:20.630676 23599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:20.634178 23599 init.cc:377] Logged in from keytab as kudu/127.8.216.188@KRBTEST.COM (short username kudu)
I20260807 19:34:20.635977 23599 webserver.cc:460] Webserver started at http://127.8.216.188:46193/ using document root <none> and password file <none>
I20260807 19:34:20.636816 23599 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:20.636979 23599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:20.637310 23599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:20.640645 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data/instance:
uuid: "7078c3c7e0084c0b8b808922d3462523"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "69c326cc44f02c15faecdba3c907372c"
server_key_iv: "3b41f480dceed55d43c7e64585b547f5"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.641656 23599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal/instance:
uuid: "7078c3c7e0084c0b8b808922d3462523"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "69c326cc44f02c15faecdba3c907372c"
server_key_iv: "3b41f480dceed55d43c7e64585b547f5"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.647444 23599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 19:34:20.651242 23615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:20.652838 23599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:34:20.653050 23599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "7078c3c7e0084c0b8b808922d3462523"
format_stamp: "Formatted at 2026-08-07 19:34:20 on dist-test-slave-8w9v"
server_key: "69c326cc44f02c15faecdba3c907372c"
server_key_iv: "3b41f480dceed55d43c7e64585b547f5"
server_key_version: "encryptionkey@0"
I20260807 19:34:20.653280 23599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:20.686758 23599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:20.691427 23599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:20.691757 23599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:20.712757 23599 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.188:35373
I20260807 19:34:20.712795 23670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.188:35373 every 8 connection(s)
I20260807 19:34:20.714284 23599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
I20260807 19:34:20.718289 23671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:20.724113 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23599
I20260807 19:34:20.724364 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131039239543-9058-0/minicluster-data/master-2/wal/instance
I20260807 19:34:20.724992 9058 external_mini_cluster.cc:1428] Setting key 43e90ce66eda063fd0c6f189e32d1d06
I20260807 19:34:20.726305 23671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:20.729705 23508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 4
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:34:20.752677 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.729781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:43425 (local address 127.0.0.1:44378)
0807 19:34:20.730795 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.730847 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.731642 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.732459 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.732471 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.733610 (+ 1139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.734573 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.734599 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.736047 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.736052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.737603 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.737619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.738954 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.738961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.739139 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.739831 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.739866 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.743162 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.747165 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.747176 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.747189 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.748070 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.748593 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.748598 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.748602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.748879 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.749692 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.749702 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.750195 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.750541 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.751141 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":85,"thread_start_us":573,"threads_started":1}
I20260807 19:34:20.752769 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.731677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60912)
0807 19:34:20.731955 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.731976 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.732108 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.735191 (+ 3083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.735197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.735704 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.735918 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.735932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.737836 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.737841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.739315 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.739335 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.740258 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.740263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.740471 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.741105 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.741155 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.744349 (+ 3194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.749504 (+ 5155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.749510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.749515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.750078 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.750533 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.750539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.750542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.750707 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.751659 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.751666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.751854 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.752038 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.752388 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260807 19:34:20.752705 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.727409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.8.216.188:35373)
0807 19:34:20.728550 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.728563 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.728607 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:34:20.731531 (+ 2924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.731563 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.731949 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.732204 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.733999 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.735803 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.737888 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.738450 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.738656 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.744177 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.744196 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.744237 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.744288 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.748371 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.749242 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.749255 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.749263 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.749421 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.749931 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.749937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.749941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.750430 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.750771 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.751126 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.751416 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":606,"threads_started":1}
I20260807 19:34:20.753324 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.731868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.8.216.188:35373)
0807 19:34:20.734859 (+ 2991us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.734865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.734890 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:34:20.734937 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.734943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.735014 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.735120 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.736177 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.737655 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.739490 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.740036 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.740256 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.745299 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.745309 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.745315 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.745355 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.749357 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.750209 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.750217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.750223 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.750377 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.750845 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.750850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.750853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.751231 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.751372 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.752205 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.752463 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2829,"thread_start_us":2497,"threads_started":1}
I20260807 19:34:20.754130 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.729702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.8.216.190:43425)
0807 19:34:20.731496 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.731501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.731553 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:34:20.732056 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.732065 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.732199 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.734005 (+ 1806us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.734777 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.735809 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.737764 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.738511 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.738646 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.743403 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.743412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.743417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.743466 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.746996 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.748246 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.748254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.748260 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.748424 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.749043 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.749049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.749052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.749531 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.749705 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.753527 (+ 3822us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.753763 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1679,"thread_start_us":1526,"threads_started":1}
I20260807 19:34:20.756174 23671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:20.761144 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: Bootstrap starting.
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:34:20.765885 23508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:20.767537 23508 log.cc:824] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:20.771258 23508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: No bootstrap required, opened a new log
I20260807 19:34:20.772838 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.757123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:38079 (local address 127.0.0.1:60988)
0807 19:34:20.757304 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.757327 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.757447 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.757768 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.757772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.758185 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.758344 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.758353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.759907 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.759911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.760876 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.760889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.761904 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.761910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.762137 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.762873 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.762900 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.765348 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.769796 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.769800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.769804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.770221 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.770725 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.770730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.770734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.770939 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.771609 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.771615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.771833 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.772021 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.772433 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:34:20.772852 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.757211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.8.216.189:38079)
0807 19:34:20.757432 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.757438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.757460 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:34:20.757532 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.757540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.757618 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.757743 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.758578 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.759727 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.761033 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.761725 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.761896 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.765589 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.765599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.765607 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.765660 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.769625 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.770376 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.770381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.770384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.770570 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.771087 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.771093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.771096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.771464 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.771624 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.772214 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.772465 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:20.774670 23671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:34:20.781477 23508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.782090 23508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:20.782263 23508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b66e801b7d404c910319b7c7cae22e, State: Initialized, Role: FOLLOWER
I20260807 19:34:20.783156 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.762510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.8.216.190:43425)
0807 19:34:20.762745 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.762752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.762874 (+ 122us) server_negotiation.cc:407] Connection header received
0807 19:34:20.763070 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.763076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.763172 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.763295 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.765196 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.766097 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.767633 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.768767 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.768933 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.775120 (+ 6187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.775129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.775135 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.775185 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.779507 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.780593 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.780602 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.780606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.780794 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.781308 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.781314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.781317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.781786 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.782197 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.782480 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.782727 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:34:20.783090 23508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.784307 23680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:34:20.784701 23680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.785928 23508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:20.794179 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.775507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60920)
0807 19:34:20.775700 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.775722 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.775848 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.776424 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.776429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.776956 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.777192 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.777206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.778663 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.778668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.779981 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.779994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.780848 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.780855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.781145 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.781810 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.781838 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.785680 (+ 3842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.791377 (+ 5697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.791383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.791387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.791809 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.792280 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.792285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.792288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.792432 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.793031 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.793036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.793211 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.793391 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.793739 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":30}
I20260807 19:34:20.794617 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.775704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.8.216.188:35373)
0807 19:34:20.775890 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.775896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.775914 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:34:20.775986 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.775992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.776063 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.776178 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.777362 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.778503 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.780161 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.780671 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.780828 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.785911 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.785921 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.785926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.785983 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.791215 (+ 5232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.791943 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.791950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.791954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.792087 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.792544 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.792552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.792556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.792900 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.793059 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.794010 (+ 951us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.794265 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":55}
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:34:20.803606 23671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: Bootstrap starting.
I20260807 19:34:20.805259 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.785891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.8.216.189:38079)
0807 19:34:20.786097 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.786105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.786256 (+ 151us) server_negotiation.cc:407] Connection header received
0807 19:34:20.786514 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.786520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.786626 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.786730 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.788904 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.790511 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.793286 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.793978 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.794808 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.799267 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.799273 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.799276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.799321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.802215 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.803053 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.803058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.803061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.803213 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.803661 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.803665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.803667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.803961 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.804299 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.804636 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.804902 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:34:20.810366 23671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:20.811818 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:34:20.811985 23671 log.cc:824] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:20.815799 23671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: No bootstrap required, opened a new log
W20260807 19:34:20.816255 23461 tablet.cc:2369] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:20.815441 23438 master_runner.cc:186] Error getting master registration for 127.8.216.189:38079: OK, instance_id { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" instance_seqno: 1786131260134664 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:34:20.829183 23438 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:34:20.831544 23671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.832080 23671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:20.832278 23671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7078c3c7e0084c0b8b808922d3462523, State: Initialized, Role: FOLLOWER
I20260807 19:34:20.833039 23671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
W20260807 19:34:20.833271 23691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:34:20.833436 23691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:34:20.841596 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.842592 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.843566 23671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:20.844547 23680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:20.844913 23680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.847071 23680 leader_election.cc:288] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1df932e760234a6dbaae37c180acc3bc (127.8.216.189:38079), 7078c3c7e0084c0b8b808922d3462523 (127.8.216.188:35373)
I20260807 19:34:20.849689 23645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21b66e801b7d404c910319b7c7cae22e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7078c3c7e0084c0b8b808922d3462523" is_pre_election: true
I20260807 19:34:20.850625 23645 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21b66e801b7d404c910319b7c7cae22e in term 0.
I20260807 19:34:20.851862 23458 leader_election.cc:302] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21b66e801b7d404c910319b7c7cae22e, 7078c3c7e0084c0b8b808922d3462523; no voters:
I20260807 19:34:20.852741 23680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:34:20.853103 23680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:34:20.853389 23680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:20.856083 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.825675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.8.216.189:38079)
0807 19:34:20.825882 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.825888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.835654 (+ 9766us) server_negotiation.cc:407] Connection header received
0807 19:34:20.836299 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.836306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.836396 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.836477 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.839346 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.840532 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.843210 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.844010 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.844172 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.847826 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.847833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.847840 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.847893 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.852099 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.853230 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.853239 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.853244 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.853426 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.853969 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.853974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.853977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.854409 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.854659 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.855368 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.855642 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:34:20.859076 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.819608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.8.216.190:43425)
0807 19:34:20.819813 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.819821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.830035 (+ 10214us) server_negotiation.cc:407] Connection header received
0807 19:34:20.830383 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.830393 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.830519 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.830615 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.833793 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.835347 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.838511 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.839267 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.839405 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.849560 (+ 10155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.849570 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.849575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.849626 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.854732 (+ 5106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.855674 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.855680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.855684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.855849 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.856437 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.856442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.856445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.856851 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.857194 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.858479 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.858744 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:34:20.860980 23680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.864588 23590 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } attempt: 5
I20260807 19:34:20.871457 23678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.830214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.8.216.188:35373)
0807 19:34:20.830430 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.830436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.838363 (+ 7927us) server_negotiation.cc:407] Connection header received
0807 19:34:20.840817 (+ 2454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.840823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.840926 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.841011 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.846133 (+ 5122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.847370 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.850489 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.851309 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.851426 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.862697 (+ 11271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.862705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.862710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.862762 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.867504 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.868629 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.868636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.868640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.868853 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.869361 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.869366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.869369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.869720 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.870038 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.870665 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.870920 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 19:34:20 dist-test-slave-8w9v krb5kdc[23422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131260, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:34:20.876382 23680 leader_election.cc:288] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [CANDIDATE]: Term 1 election: Requested vote from peers 1df932e760234a6dbaae37c180acc3bc (127.8.216.189:38079), 7078c3c7e0084c0b8b808922d3462523 (127.8.216.188:35373)
I20260807 19:34:20.876523 23645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21b66e801b7d404c910319b7c7cae22e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7078c3c7e0084c0b8b808922d3462523"
I20260807 19:34:20.876834 23645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:20.880954 23645 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21b66e801b7d404c910319b7c7cae22e in term 1.
I20260807 19:34:20.881667 23458 leader_election.cc:302] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21b66e801b7d404c910319b7c7cae22e, 7078c3c7e0084c0b8b808922d3462523; no voters:
I20260807 19:34:20.882181 23680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:20.884462 23680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [term 1 LEADER]: Becoming Leader. State: Replica: 21b66e801b7d404c910319b7c7cae22e, State: Running, Role: LEADER
I20260807 19:34:20.885380 23674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.866476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.8.216.188:35373)
0807 19:34:20.866644 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:20.866650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:20.866671 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:34:20.866747 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:20.866752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:20.866832 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:20.866929 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:20.867977 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.869117 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.870910 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.871641 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.871764 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.876601 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:20.876608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:20.876612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:20.876654 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:20.880889 (+ 4235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.882004 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.882009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.882013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.882167 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:20.882776 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:20.882781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:20.882784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:20.883217 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:20.883378 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:20.884784 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0807 19:34:20.885035 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 19:34:20.885185 23680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.884583 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:20.865606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:35373 (local address 127.0.0.1:60936)
0807 19:34:20.866056 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:20.866077 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:20.866205 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:20.866972 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:20.866978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:20.867522 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:20.867748 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.867762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.869275 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.869284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.870740 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:20.870754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:20.872291 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:20.872306 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:20.872579 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:20.873275 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:20.873310 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:20.876352 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:20.881211 (+ 4859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.881219 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.881225 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.881842 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.882379 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:20.882383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:20.882387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:20.882564 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:20.883456 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:20.883464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:20.883642 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:34:20.883811 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 19:34:20.884156 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313}
I20260807 19:34:20.892336 23590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: Bootstrap starting.
I20260807 19:34:20.893939 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b66e801b7d404c910319b7c7cae22e. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.894440 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:20.896092 23708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:20.897337 23590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:20.898936 23590 log.cc:824] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:20.906165 23590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: No bootstrap required, opened a new log
I20260807 19:34:20.908059 23708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:20.917960 23590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.918599 23590 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:20.918777 23590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1df932e760234a6dbaae37c180acc3bc, State: Initialized, Role: FOLLOWER
I20260807 19:34:20.919660 23590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } }
I20260807 19:34:20.926862 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.929945 23590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:20.929627 23565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21b66e801b7d404c910319b7c7cae22e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1df932e760234a6dbaae37c180acc3bc" is_pre_election: true
I20260807 19:34:20.930452 23565 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21b66e801b7d404c910319b7c7cae22e in term 0.
I20260807 19:34:20.929702 23564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21b66e801b7d404c910319b7c7cae22e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1df932e760234a6dbaae37c180acc3bc"
I20260807 19:34:20.929634 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.931509 23564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:20.935578 23564 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21b66e801b7d404c910319b7c7cae22e in term 1.
W20260807 19:34:20.944339 23622 tablet.cc:2369] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:20.947857 23645 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [term 1 FOLLOWER]: Refusing update from remote peer 21b66e801b7d404c910319b7c7cae22e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:20.948350 23564 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [term 1 FOLLOWER]: Refusing update from remote peer 21b66e801b7d404c910319b7c7cae22e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:20.950105 23680 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:20.950927 23724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:20.965602 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b66e801b7d404c910319b7c7cae22e. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.965894 23715 mvcc.cc:205] Tried to move back new op lower bound from 7315993644822728704 to 7315993644592361472. Current Snapshot: MvccSnapshot[applied={T|T < 7315993644822728704}]
I20260807 19:34:20.966027 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.971146 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.971469 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.979624 23724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.980103 23724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:20.980791 23708 catalog_manager.cc:1347] Generated new cluster ID: a91cec216e524660aa2b97da4d69f7c7
I20260807 19:34:20.980901 23708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:20.981608 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b66e801b7d404c910319b7c7cae22e. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.981961 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:20.985126 23735 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:34:20.989323 23680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:20.989717 23680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:20.990633 23740 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:34:20.994386 23731 mvcc.cc:205] Tried to move back new op lower bound from 7315993644822728704 to 7315993644592361472. Current Snapshot: MvccSnapshot[applied={T|T < 7315993644822728704}]
I20260807 19:34:21.004734 23708 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:34:21.008284 23708 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:21.008277 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21b66e801b7d404c910319b7c7cae22e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21b66e801b7d404c910319b7c7cae22e" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 43425 } } peers { permanent_uuid: "1df932e760234a6dbaae37c180acc3bc" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 38079 } } peers { permanent_uuid: "7078c3c7e0084c0b8b808922d3462523" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 35373 } } }
I20260807 19:34:21.008617 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:34:21.021685 23708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21b66e801b7d404c910319b7c7cae22e: Generated new TSK 0
I20260807 19:34:21.022919 23708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:34:21.990078 23735 catalog_manager.cc:1259] Loaded cluster ID: a91cec216e524660aa2b97da4d69f7c7
I20260807 19:34:21.990202 23735 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: loading cluster ID for follower catalog manager: success
I20260807 19:34:21.994421 23735 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: acquiring CA information for follower catalog manager: success
I20260807 19:34:21.996411 23740 catalog_manager.cc:1259] Loaded cluster ID: a91cec216e524660aa2b97da4d69f7c7
I20260807 19:34:21.996618 23740 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: loading cluster ID for follower catalog manager: success
I20260807 19:34:21.997382 23735 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1df932e760234a6dbaae37c180acc3bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:34:21.999743 23740 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: acquiring CA information for follower catalog manager: success
I20260807 19:34:22.001784 23740 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7078c3c7e0084c0b8b808922d3462523: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:34:22.120748 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23438
I20260807 19:34:22.122201 23503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:22.317670 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23438
I20260807 19:34:22.357187 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23515
I20260807 19:34:22.358810 23585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:22.473762 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23515
I20260807 19:34:22.496917 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23599
I20260807 19:34:22.498162 23666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:22.609045 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23599
2026-08-07T19:34:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4988 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:34:22 dist-test-slave-8w9v krb5kdc[23755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:34:22 dist-test-slave-8w9v krb5kdc[23755](info): set up 2 sockets
Aug 07 19:34:22 dist-test-slave-8w9v krb5kdc[23755](info): commencing operation
krb5kdc: starting...
W20260807 19:34:23.803951 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:34:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:34:24.129326 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38719
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:38719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:24.499051 23769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:24.499439 23769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:24.499609 23769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:24.507277 23769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:24.507418 23769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:24.507468 23769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:24.507592 23769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:24.507660 23769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:24.518429 23769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:38719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:24.519966 23769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:24.521451 23769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:24.535089 23777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:24.535097 23774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:24.535632 23769 server_base.cc:1034] running on GCE node
W20260807 19:34:24.535066 23775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:24.537068 23769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:24.538821 23769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:24.540091 23769 hybrid_clock.cc:648] HybridClock initialized: now 1786131264540038 us; error 40 us; skew 500 ppm
I20260807 19:34:24.540633 23769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131264, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:24.544173 23769 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:24.545954 23769 webserver.cc:460] Webserver started at http://127.8.216.190:36993/ using document root <none> and password file <none>
I20260807 19:34:24.546715 23769 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:24.546877 23769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:24.547226 23769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:24.550645 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "7296fa77bcfb4f589cafbd0d2e7ef601"
format_stamp: "Formatted at 2026-08-07 19:34:24 on dist-test-slave-8w9v"
server_key: "48e9c08f7effd79f0b349d6ebd648cd4"
server_key_iv: "f3a6280b2ddb2ab8eb8f873c6d0709e3"
server_key_version: "encryptionkey@0"
I20260807 19:34:24.551512 23769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "7296fa77bcfb4f589cafbd0d2e7ef601"
format_stamp: "Formatted at 2026-08-07 19:34:24 on dist-test-slave-8w9v"
server_key: "48e9c08f7effd79f0b349d6ebd648cd4"
server_key_iv: "f3a6280b2ddb2ab8eb8f873c6d0709e3"
server_key_version: "encryptionkey@0"
I20260807 19:34:24.557511 23769 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 19:34:24.561787 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:24.563354 23769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:34:24.563613 23769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "7296fa77bcfb4f589cafbd0d2e7ef601"
format_stamp: "Formatted at 2026-08-07 19:34:24 on dist-test-slave-8w9v"
server_key: "48e9c08f7effd79f0b349d6ebd648cd4"
server_key_iv: "f3a6280b2ddb2ab8eb8f873c6d0709e3"
server_key_version: "encryptionkey@0"
I20260807 19:34:24.563903 23769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:24.587067 23769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:24.592577 23769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:24.592983 23769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:24.616361 23769 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:38719
I20260807 19:34:24.616369 23837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:38719 every 8 connection(s)
I20260807 19:34:24.618093 23769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:24.622189 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23769
I20260807 19:34:24.622449 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:34:24.622745 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:24.623411 9058 external_mini_cluster.cc:1428] Setting key 62c3eaa554d5fdb5211eb744974ea6fe
I20260807 19:34:24.638757 23838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601: Bootstrap starting.
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:24.642277 23838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:24.643600 23838 log.cc:824] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:24.647045 23838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601: No bootstrap required, opened a new log
I20260807 19:34:24.650921 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:24.625619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.8.216.190:38719)
0807 19:34:24.626686 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:24.626704 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:24.626753 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:34:24.628693 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:24.628721 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:24.629218 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:24.629628 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:24.631746 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:24.633358 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:24.635427 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:24.636481 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:24.636732 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:24.641839 (+ 5107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:24.641849 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:24.641866 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:24.641915 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:24.645789 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:24.646734 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:24.646741 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:24.646748 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:24.646952 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:24.647490 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:24.647496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:24.647510 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:24.648100 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:24.648525 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:24.649133 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 19:34:24.649671 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":557,"threads_started":1}
I20260807 19:34:24.655846 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:34:24.656860 23838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } }
I20260807 19:34:24.657300 23838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:24.657424 23838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7296fa77bcfb4f589cafbd0d2e7ef601, State: Initialized, Role: FOLLOWER
I20260807 19:34:24.658097 23838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } }
I20260807 19:34:24.658380 23838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:24.658552 23838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:24.658802 23838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:24.661527 23838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } }
I20260807 19:34:24.662076 23838 leader_election.cc:302] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7296fa77bcfb4f589cafbd0d2e7ef601; no voters:
I20260807 19:34:24.662712 23838 leader_election.cc:288] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:34:24.662895 23843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:24.663671 23843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [term 1 LEADER]: Becoming Leader. State: Replica: 7296fa77bcfb4f589cafbd0d2e7ef601, State: Running, Role: LEADER
I20260807 19:34:24.664470 23843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } }
I20260807 19:34:24.665066 23838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:24.667719 23845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7296fa77bcfb4f589cafbd0d2e7ef601. Latest consensus state: current_term: 1 leader_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } } }
I20260807 19:34:24.667692 23844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7296fa77bcfb4f589cafbd0d2e7ef601" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38719 } } }
I20260807 19:34:24.668323 23845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:24.668390 23844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:24.674939 23850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:24.679694 23850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:24.688704 23850 catalog_manager.cc:1347] Generated new cluster ID: 8679643b03824b80bc173f2cd6942439
I20260807 19:34:24.688858 23850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:24.705137 23850 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:34:24.706664 23850 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:24.714716 23850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7296fa77bcfb4f589cafbd0d2e7ef601: Generated new TSK 0
I20260807 19:34:24.715812 23850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:34:24.770104 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23769
I20260807 19:34:24.771623 23833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:24.874044 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23769
2026-08-07T19:34:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2259 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23868](info): set up 2 sockets
Aug 07 19:34:24 dist-test-slave-8w9v krb5kdc[23868](info): commencing operation
krb5kdc: starting...
W20260807 19:34:26.064913 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:34:26 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:34:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:34:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:34:26.393596 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:26.779676 23881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:26.780037 23881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:26.780241 23881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:26.788084 23881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:26.788250 23881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:26.788419 23881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:26.788532 23881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:26.788614 23881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:26.799211 23881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:26.800766 23881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:26.802201 23881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:26.816010 23886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:26.815884 23888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:26.816236 23890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:26.817476 23881 server_base.cc:1034] running on GCE node
I20260807 19:34:26.818503 23881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:26.820261 23881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:26.821560 23881 hybrid_clock.cc:648] HybridClock initialized: now 1786131266821514 us; error 47 us; skew 500 ppm
I20260807 19:34:26.821993 23881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:26 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:26.825681 23881 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:26.827527 23881 webserver.cc:460] Webserver started at http://127.8.216.190:43357/ using document root <none> and password file <none>
I20260807 19:34:26.828425 23881 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:26.828588 23881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:26.828926 23881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:26.832412 23881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "073f78f2402a4bbbba28d53013f70faf"
format_stamp: "Formatted at 2026-08-07 19:34:26 on dist-test-slave-8w9v"
server_key: "962d3b2b28712c4d4386490abe67badc"
server_key_iv: "ea617d8467eab5db44abbcbf15d63c88"
server_key_version: "encryptionkey@0"
I20260807 19:34:26.833276 23881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "073f78f2402a4bbbba28d53013f70faf"
format_stamp: "Formatted at 2026-08-07 19:34:26 on dist-test-slave-8w9v"
server_key: "962d3b2b28712c4d4386490abe67badc"
server_key_iv: "ea617d8467eab5db44abbcbf15d63c88"
server_key_version: "encryptionkey@0"
I20260807 19:34:26.839526 23881 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:34:26.843822 23898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:26.845579 23881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:34:26.845853 23881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "073f78f2402a4bbbba28d53013f70faf"
format_stamp: "Formatted at 2026-08-07 19:34:26 on dist-test-slave-8w9v"
server_key: "962d3b2b28712c4d4386490abe67badc"
server_key_iv: "ea617d8467eab5db44abbcbf15d63c88"
server_key_version: "encryptionkey@0"
I20260807 19:34:26.846114 23881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:26.880301 23881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:26.886397 23881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:26.886770 23881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:26.909060 23881 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36933
I20260807 19:34:26.909111 23951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36933 every 8 connection(s)
I20260807 19:34:26.910576 23881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:26.914810 23952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:26.918416 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23881
I20260807 19:34:26.918705 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:34:26.919571 9058 external_mini_cluster.cc:1428] Setting key bc071101025b066769ac6320944d90f6
I20260807 19:34:26.928771 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap starting.
I20260807 19:34:26.932212 23952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:26.933367 23952 log.cc:824] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:34:26 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:26.936877 23952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: No bootstrap required, opened a new log
I20260807 19:34:26.945597 23952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:26.946066 23952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:26.946201 23952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Initialized, Role: FOLLOWER
I20260807 19:34:26.946751 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:26.922106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.8.216.190:36933)
0807 19:34:26.923030 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:26.923044 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:26.923090 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:34:26.925142 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:26.925169 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:26.925669 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:26.926167 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:26.927913 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:26.929564 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:26.931171 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:26.931947 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:26.932219 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:26.937366 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:26.937375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:26.937395 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:26.937444 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:26.941325 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:26.942291 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:26.942299 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:26.942306 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:26.942533 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:26.943030 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:26.943044 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:26.943047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:26.943764 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:26.944250 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:26.944835 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 19:34:26.945405 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":423,"threads_started":1}
I20260807 19:34:26.946929 23952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:26.947258 23952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:26.947429 23952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:26.947647 23952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:26.950601 23952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:26.951185 23952 leader_election.cc:302] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 073f78f2402a4bbbba28d53013f70faf; no voters:
I20260807 19:34:26.951841 23952 leader_election.cc:288] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:34:26.952024 23957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:26.952937 23957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 LEADER]: Becoming Leader. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Running, Role: LEADER
I20260807 19:34:26.953610 23957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:26.954236 23952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:26.956674 23958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:26.957103 23958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:26.958251 23963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:26.958132 23959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073f78f2402a4bbbba28d53013f70faf. Latest consensus state: current_term: 1 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:26.958587 23959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:26.967868 23963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:26.981285 23963 catalog_manager.cc:1347] Generated new cluster ID: 6788afe0604649efb9a89a553bd04531
I20260807 19:34:26.981467 23963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:27.003134 23963 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:34:27.004848 23963 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:27.013593 23963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Generated new TSK 0
I20260807 19:34:27.014629 23963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:34:27.094303 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:27.470372 23980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:27.470678 23980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:27.470860 23980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:27.478493 23980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:27.478646 23980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:27.478835 23980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:27.489823 23980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:27.491285 23980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:27.492875 23980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:27.506901 23989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:27.506839 23986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:27.507072 23987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:27.508426 23980 server_base.cc:1034] running on GCE node
I20260807 19:34:27.509121 23980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:27.510237 23980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:27.511487 23980 hybrid_clock.cc:648] HybridClock initialized: now 1786131267511410 us; error 60 us; skew 500 ppm
I20260807 19:34:27.511811 23980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:27 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:27.515494 23980 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:27.517433 23980 webserver.cc:460] Webserver started at http://127.8.216.129:33197/ using document root <none> and password file <none>
I20260807 19:34:27.518239 23980 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:27.518402 23980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:27.518766 23980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:27.522076 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
format_stamp: "Formatted at 2026-08-07 19:34:27 on dist-test-slave-8w9v"
server_key: "725e44ba96a5c374186eac3fc9a6dd93"
server_key_iv: "7b5e171b9629bc11b7765e2ee69b9f39"
server_key_version: "encryptionkey@0"
I20260807 19:34:27.522967 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
format_stamp: "Formatted at 2026-08-07 19:34:27 on dist-test-slave-8w9v"
server_key: "725e44ba96a5c374186eac3fc9a6dd93"
server_key_iv: "7b5e171b9629bc11b7765e2ee69b9f39"
server_key_version: "encryptionkey@0"
I20260807 19:34:27.529081 23980 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:34:27.533169 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:27.534698 23980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:34:27.534909 23980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
format_stamp: "Formatted at 2026-08-07 19:34:27 on dist-test-slave-8w9v"
server_key: "725e44ba96a5c374186eac3fc9a6dd93"
server_key_iv: "7b5e171b9629bc11b7765e2ee69b9f39"
server_key_version: "encryptionkey@0"
I20260807 19:34:27.535153 23980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:27.560573 23980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:27.565514 23980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:27.565843 23980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:27.566926 23980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:27.568488 23980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:27.568602 23980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:27.568795 23980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:27.568872 23980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:27.605933 23980 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38695
I20260807 19:34:27.606033 24124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38695 every 8 connection(s)
I20260807 19:34:27.607347 23980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:27.610153 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 23980
I20260807 19:34:27.610427 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:34:27.611210 9058 external_mini_cluster.cc:1428] Setting key 58746e90bc8fe95e32448615e38cf7b9
Aug 07 19:34:27 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:27.633911 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:27.611751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:41025 (local address 127.8.216.190:36933)
0807 19:34:27.612670 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:27.612679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:27.612706 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:34:27.615164 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:27.615171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:27.615281 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:27.615425 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:27.617378 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:27.618173 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:27.620039 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:27.620594 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:27.620773 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:27.625509 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:27.625519 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:27.625526 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:27.625586 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:27.629759 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:27.630749 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:27.630759 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:27.630765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:27.630961 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:27.631450 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:27.631458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:27.631464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:27.631935 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:27.632114 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:27.633020 (+ 906us) server_negotiation.cc:299] Negotiation successful
0807 19:34:27.633323 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":674,"threads_started":1}
I20260807 19:34:27.634477 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:27.611156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:41025)
0807 19:34:27.612210 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:27.612295 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:27.614838 (+ 2543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:27.615705 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:27.615717 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:27.616448 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:27.617124 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:27.617153 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:27.618384 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:27.618389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:27.619819 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:27.619836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:27.620918 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:27.620927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:27.621221 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:27.622079 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:27.622108 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:27.625124 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:27.629998 (+ 4874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:27.630007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:27.630024 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:27.630559 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:27.631093 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:27.631096 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:27.631099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:27.631300 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:27.632075 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:27.632081 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:27.632568 (+ 487us) client_negotiation.cc:769] Sending connection context
0807 19:34:27.632813 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 19:34:27.633259 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":359,"threads_started":1}
I20260807 19:34:27.636423 24125 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:27.636790 24125 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:27.637611 24125 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:27.640312 23915 ts_manager.cc:194] Registered new tserver with Master: eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695)
I20260807 19:34:27.641901 23915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:41025
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:34:27.685132 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:28.078751 24131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:28.079087 24131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:28.079267 24131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:28.087265 24131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:28.087428 24131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:28.087666 24131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:28.099064 24131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:28.100667 24131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:28.102170 24131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:28.117668 24140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:28.117617 24138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:28.117875 24137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:28.118717 24131 server_base.cc:1034] running on GCE node
I20260807 19:34:28.119434 24131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:28.120543 24131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:28.121824 24131 hybrid_clock.cc:648] HybridClock initialized: now 1786131268121762 us; error 56 us; skew 500 ppm
I20260807 19:34:28.122241 24131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:28 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131268, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:28.131004 24131 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:28.132833 24131 webserver.cc:460] Webserver started at http://127.8.216.130:35803/ using document root <none> and password file <none>
I20260807 19:34:28.133653 24131 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:28.133821 24131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:28.134171 24131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:28.137552 24131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "06833fae4c304fd091d5e90d3d214bd5"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "27eaae018a3115e11b8ae30b8ac0062f"
server_key_iv: "4f81ed7c5baeec504c20c20d184c5b24"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.138447 24131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "06833fae4c304fd091d5e90d3d214bd5"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "27eaae018a3115e11b8ae30b8ac0062f"
server_key_iv: "4f81ed7c5baeec504c20c20d184c5b24"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.144845 24131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:34:28.148880 24147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.150363 24131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:34:28.150588 24131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "06833fae4c304fd091d5e90d3d214bd5"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "27eaae018a3115e11b8ae30b8ac0062f"
server_key_iv: "4f81ed7c5baeec504c20c20d184c5b24"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.150830 24131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:28.177493 24131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:28.182358 24131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:28.182737 24131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:28.183872 24131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:28.185535 24131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:28.185660 24131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.185827 24131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:28.185914 24131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.223558 24131 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:45135
I20260807 19:34:28.223616 24267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:45135 every 8 connection(s)
I20260807 19:34:28.225021 24131 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:28.229449 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24131
I20260807 19:34:28.229666 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:34:28.230297 9058 external_mini_cluster.cc:1428] Setting key 0dc0842ba01b3fcb31a0c921a0ea2c05
Aug 07 19:34:28 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131268, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:28.250732 24271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.227878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:33531 (local address 127.8.216.190:36933)
0807 19:34:28.228472 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.228477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.229742 (+ 1265us) server_negotiation.cc:407] Connection header received
0807 19:34:28.232315 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.232340 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.232460 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.232571 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:28.234370 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.235634 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.237579 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.238381 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.238535 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.242846 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:28.242856 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:28.242863 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:28.242932 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:28.246477 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.247735 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.247744 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.247750 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.247934 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.248548 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.248552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.248554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.248857 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:28.249001 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.250056 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.250311 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":341,"threads_started":1}
I20260807 19:34:28.252338 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.228543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:33531)
0807 19:34:28.229533 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:28.229593 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:28.231942 (+ 2349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:28.232895 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:28.232905 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:28.233506 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:28.234157 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.234177 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.235852 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.235856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.237395 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.237411 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.238728 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.238734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.238982 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.239867 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:28.239893 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:28.242496 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:28.246693 (+ 4197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:28.246707 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:28.246731 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:28.247542 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:28.248089 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:28.248096 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:28.248103 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:28.248423 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:28.249022 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:28.249035 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:28.249613 (+ 578us) client_negotiation.cc:769] Sending connection context
0807 19:34:28.249956 (+ 343us) client_negotiation.cc:240] Negotiation successful
0807 19:34:28.250682 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":336,"threads_started":1}
I20260807 19:34:28.254381 24269 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:28.254817 24269 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:28.255805 24269 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:28.257726 23915 ts_manager.cc:194] Registered new tserver with Master: 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
I20260807 19:34:28.259239 23915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:33531
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:34:28.305038 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:28.645359 24125 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
W20260807 19:34:28.690542 24276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:28.690917 24276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:28.691102 24276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:28.698824 24276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:28.699016 24276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:28.699195 24276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:28.709822 24276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:28.711309 24276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:28.713001 24276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:28.727460 24281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:28.727555 24284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:28.727516 24282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:28.728443 24276 server_base.cc:1034] running on GCE node
I20260807 19:34:28.729084 24276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:28.730142 24276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:28.731361 24276 hybrid_clock.cc:648] HybridClock initialized: now 1786131268731290 us; error 60 us; skew 500 ppm
I20260807 19:34:28.731664 24276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:28 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131268, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:28.735287 24276 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:28.737042 24276 webserver.cc:460] Webserver started at http://127.8.216.131:43957/ using document root <none> and password file <none>
I20260807 19:34:28.737821 24276 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:28.737975 24276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:28.738323 24276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:28.741674 24276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "c10b17eb8eaf813bd8278e5c24538eab"
server_key_iv: "f7e1327e4719061bbd80cf6c3c42b417"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.742533 24276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "c10b17eb8eaf813bd8278e5c24538eab"
server_key_iv: "f7e1327e4719061bbd80cf6c3c42b417"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.748581 24276 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 19:34:28.752815 24291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.754562 24276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:34:28.754738 24276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "c10b17eb8eaf813bd8278e5c24538eab"
server_key_iv: "f7e1327e4719061bbd80cf6c3c42b417"
server_key_version: "encryptionkey@0"
I20260807 19:34:28.754953 24276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:28.773101 24276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:28.777992 24276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:28.778280 24276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:28.779315 24276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:28.780879 24276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:28.780973 24276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.781080 24276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:28.781128 24276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:28.818441 24276 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:34715
I20260807 19:34:28.818602 24417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:34715 every 8 connection(s)
I20260807 19:34:28.819865 24276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:28.830324 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24276
I20260807 19:34:28.830569 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:34:28.831117 9058 external_mini_cluster.cc:1428] Setting key eb213dc1a485ab11f20da4760e79a481
Aug 07 19:34:28 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131268, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:28.844072 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.822911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:42029 (local address 127.8.216.190:36933)
0807 19:34:28.823414 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.823420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.824812 (+ 1392us) server_negotiation.cc:407] Connection header received
0807 19:34:28.826955 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.826960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.827066 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.827180 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:28.829192 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.830030 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.831928 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.832515 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.832658 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.837295 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:28.837301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:28.837305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:28.837348 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:28.840724 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.841638 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.841643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.841645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.841794 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.842244 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.842248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.842256 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.842559 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:28.842696 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.843496 (+ 800us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.843711 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":219,"threads_started":1}
I20260807 19:34:28.845086 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.823573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:42029)
0807 19:34:28.824602 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:28.824650 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:28.826601 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:28.827502 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:28.827511 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:28.828276 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:28.828957 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.828978 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.830271 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.830275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.831754 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.831768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.832849 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.832858 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.833269 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.834188 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:28.834214 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:28.836961 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:28.840905 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:28.840914 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:28.840930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:28.841478 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:28.841902 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:28.841905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:28.841907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:28.842115 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:28.842685 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:28.842691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:28.843074 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 19:34:28.843325 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 19:34:28.843780 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":353,"threads_started":1}
I20260807 19:34:28.846982 24418 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:28.847437 24418 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:28.848456 24418 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:28.850147 23914 ts_manager.cc:194] Registered new tserver with Master: 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715)
I20260807 19:34:28.851176 23914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:42029
I20260807 19:34:28.861493 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:34:28.882773 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.867078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45776 (local address 127.8.216.190:36933)
0807 19:34:28.867441 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.867450 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.867851 (+ 401us) server_negotiation.cc:407] Connection header received
0807 19:34:28.868092 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.868096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.868245 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.868371 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:28.869940 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.870708 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.872270 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.872714 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.872889 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.875821 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:28.875827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:28.875831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:28.875875 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:28.879255 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.880205 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.880211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.880221 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.880378 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:28.880883 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:28.880890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:28.880900 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:28.881320 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:28.881623 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.882059 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.882346 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:34:28.887755 23914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:34:28.895893 23914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:34:28.939371 24433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.911144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45135 (local address 127.0.0.1:49202)
0807 19:34:28.927914 (+ 16770us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:28.927940 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:28.928253 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:28.929004 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:28.929009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:28.929040 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:28.930543 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.930558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.932716 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.932721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.936184 (+ 3463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.936207 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.938365 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.938371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.938578 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.938611 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:34:28.938715 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:34:28.938935 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16605,"mutex_wait_us":102,"thread_start_us":16448,"threads_started":1}
I20260807 19:34:28.939719 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.912006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.8.216.129:38695)
0807 19:34:28.921086 (+ 9080us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.921102 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.926184 (+ 5082us) server_negotiation.cc:407] Connection header received
0807 19:34:28.926370 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.926384 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.926771 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.927069 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:28.930963 (+ 3894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.932953 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.936374 (+ 3421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.937661 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.937817 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.938093 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.939126 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.939341 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8814,"thread_start_us":8565,"threads_started":1}
I20260807 19:34:28.939991 24434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.911111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50902)
0807 19:34:28.925488 (+ 14377us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:28.925555 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:28.925870 (+ 315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:28.928858 (+ 2988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:28.928869 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:28.928909 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:28.930688 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.930700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.933178 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.933183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.936153 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.936173 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.937919 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.937928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.938151 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.938237 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 19:34:28.938401 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 19:34:28.938657 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13941,"mutex_wait_us":5,"thread_start_us":13709,"threads_started":1}
I20260807 19:34:28.939486 24436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.912619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.8.216.130:45135)
0807 19:34:28.924687 (+ 12068us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.924701 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.928082 (+ 3381us) server_negotiation.cc:407] Connection header received
0807 19:34:28.928510 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.928522 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.928821 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.929032 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:28.930808 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.932531 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.936488 (+ 3957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.938114 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.938296 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.938521 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.938863 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.939117 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11869,"thread_start_us":11722,"threads_started":1}
I20260807 19:34:28.940377 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.926108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:34715 (local address 127.0.0.1:55558)
0807 19:34:28.930392 (+ 4284us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:28.930422 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:28.930700 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:28.931797 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:28.931802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:28.931831 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:28.932349 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.932364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.934730 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.934736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.937265 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:28.937293 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.939522 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.939531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:28.939717 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.939750 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:34:28.939842 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:34:28.939994 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4043,"thread_start_us":2651,"threads_started":1}
I20260807 19:34:28.941011 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:28.926109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.8.216.131:34715)
0807 19:34:28.928490 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:28.928500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:28.930563 (+ 2063us) server_negotiation.cc:407] Connection header received
0807 19:34:28.930933 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:28.930942 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:28.931223 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:28.931436 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:28.932569 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.934509 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.937464 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:28.938990 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:28.939130 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:28.939347 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:28.940514 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0807 19:34:28.940709 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2160,"thread_start_us":1960,"threads_started":1}
I20260807 19:34:28.944504 24053 tablet_service.cc:1467] Processing CreateTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba (DEFAULT_TABLE table=test-table [id=e4be42953def4914b7a702777ba0f6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:28.944803 24344 tablet_service.cc:1467] Processing CreateTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba (DEFAULT_TABLE table=test-table [id=e4be42953def4914b7a702777ba0f6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:28.944793 24198 tablet_service.cc:1467] Processing CreateTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba (DEFAULT_TABLE table=test-table [id=e4be42953def4914b7a702777ba0f6dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:28.945928 24053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243a7ca423b945eaaf7e48dd8e4682ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:28.946368 24198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243a7ca423b945eaaf7e48dd8e4682ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:28.946430 24344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243a7ca423b945eaaf7e48dd8e4682ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:28.959391 24439 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Bootstrap starting.
I20260807 19:34:28.963634 24439 tablet_bootstrap.cc:654] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:28.965054 24439 log.cc:824] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:28.968736 24441 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Bootstrap starting.
I20260807 19:34:28.969187 24440 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Bootstrap starting.
I20260807 19:34:28.973222 24441 tablet_bootstrap.cc:654] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:28.973335 24440 tablet_bootstrap.cc:654] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:28.973531 24439 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: No bootstrap required, opened a new log
I20260807 19:34:28.973860 24439 ts_tablet_manager.cc:1397] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260807 19:34:28.974447 24441 log.cc:824] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:28.974671 24440 log.cc:824] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:28.978248 24441 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: No bootstrap required, opened a new log
I20260807 19:34:28.978590 24441 ts_tablet_manager.cc:1397] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:34:28.979050 24440 tablet_bootstrap.cc:492] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: No bootstrap required, opened a new log
I20260807 19:34:28.979341 24440 ts_tablet_manager.cc:1397] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:34:28.982718 24439 raft_consensus.cc:348] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.983193 24439 raft_consensus.cc:374] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:28.983325 24439 raft_consensus.cc:729] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb035a92774e4370bc69c57c5b3cbf3b, State: Initialized, Role: FOLLOWER
I20260807 19:34:28.984025 24439 consensus_queue.cc:260] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.986100 24439 ts_tablet_manager.cc:1428] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 19:34:28.990186 24441 raft_consensus.cc:348] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.990214 24440 raft_consensus.cc:348] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.990738 24440 raft_consensus.cc:374] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:28.990738 24441 raft_consensus.cc:374] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:28.990854 24441 raft_consensus.cc:729] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e42b690044ea9a6b57f7455f8c41e, State: Initialized, Role: FOLLOWER
I20260807 19:34:28.990852 24440 raft_consensus.cc:729] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06833fae4c304fd091d5e90d3d214bd5, State: Initialized, Role: FOLLOWER
I20260807 19:34:28.991595 24441 consensus_queue.cc:260] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.991595 24440 consensus_queue.cc:260] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:28.997074 24418 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
I20260807 19:34:28.997584 24269 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
I20260807 19:34:28.998087 24441 ts_tablet_manager.cc:1428] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260807 19:34:28.998210 24440 ts_tablet_manager.cc:1428] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Time spent starting tablet: real 0.019s user 0.009s sys 0.011s
W20260807 19:34:29.071861 24419 tablet.cc:2369] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:29.089788 24445 raft_consensus.cc:482] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:29.090153 24445 raft_consensus.cc:504] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:29.092049 24445 leader_election.cc:288] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135), 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715)
I20260807 19:34:29.099078 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.092630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:57745 (local address 127.8.216.131:34715)
0807 19:34:29.092896 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.092904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.093515 (+ 611us) server_negotiation.cc:407] Connection header received
0807 19:34:29.093763 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.093767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.093866 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.093958 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:29.094510 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.095294 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.097270 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.098048 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.098274 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.098362 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.098642 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.098785 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 19:34:29.099151 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.092427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:34715 (local address 127.8.216.129:57745)
0807 19:34:29.093348 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.093364 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:29.093591 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:29.093987 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:29.093992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:29.094019 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:29.094359 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:29.094369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.095435 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.095443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:29.097081 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:29.097095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.098282 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.098289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:29.098471 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.098496 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:34:29.098610 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:34:29.098776 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":555,"threads_started":1}
I20260807 19:34:29.099579 24436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.092649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60783 (local address 127.8.216.130:45135)
0807 19:34:29.093135 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.093144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.093876 (+ 732us) server_negotiation.cc:407] Connection header received
0807 19:34:29.094088 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.094092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.094159 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.094291 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:29.094957 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.095708 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.097649 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.098489 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.098722 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.098812 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.099137 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.099289 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260807 19:34:29.099651 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.092757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45135 (local address 127.8.216.129:60783)
0807 19:34:29.093748 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.093764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:29.093901 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:29.094317 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:29.094323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:29.094356 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:29.094758 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:29.094768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.095890 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.095905 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:29.097454 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:29.097468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.098760 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.098768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:29.098955 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.098981 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:34:29.099107 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:34:29.099257 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":598,"threads_started":1}
I20260807 19:34:29.100682 24366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243a7ca423b945eaaf7e48dd8e4682ba" candidate_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" is_pre_election: true
I20260807 19:34:29.100868 24220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243a7ca423b945eaaf7e48dd8e4682ba" candidate_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06833fae4c304fd091d5e90d3d214bd5" is_pre_election: true
I20260807 19:34:29.101411 24366 raft_consensus.cc:2454] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb035a92774e4370bc69c57c5b3cbf3b in term 0.
I20260807 19:34:29.101433 24220 raft_consensus.cc:2454] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb035a92774e4370bc69c57c5b3cbf3b in term 0.
I20260807 19:34:29.102274 24000 leader_election.cc:302] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e, eb035a92774e4370bc69c57c5b3cbf3b; no voters:
I20260807 19:34:29.102754 24445 raft_consensus.cc:2781] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:34:29.102943 24445 raft_consensus.cc:482] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:34:29.103060 24445 raft_consensus.cc:3037] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:29.105829 24445 raft_consensus.cc:504] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:29.106498 24445 leader_election.cc:288] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 1 election: Requested vote from peers 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135), 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715)
I20260807 19:34:29.107056 24220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243a7ca423b945eaaf7e48dd8e4682ba" candidate_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06833fae4c304fd091d5e90d3d214bd5"
I20260807 19:34:29.107079 24366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243a7ca423b945eaaf7e48dd8e4682ba" candidate_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
I20260807 19:34:29.107318 24366 raft_consensus.cc:3037] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:29.107326 24220 raft_consensus.cc:3037] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:29.109767 24366 raft_consensus.cc:2454] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb035a92774e4370bc69c57c5b3cbf3b in term 1.
I20260807 19:34:29.109833 24220 raft_consensus.cc:2454] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb035a92774e4370bc69c57c5b3cbf3b in term 1.
I20260807 19:34:29.110350 24000 leader_election.cc:302] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e42b690044ea9a6b57f7455f8c41e, eb035a92774e4370bc69c57c5b3cbf3b; no voters:
W20260807 19:34:29.110515 24126 tablet.cc:2369] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:29.110736 24445 raft_consensus.cc:2781] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:29.111299 24445 raft_consensus.cc:686] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 1 LEADER]: Becoming Leader. State: Replica: eb035a92774e4370bc69c57c5b3cbf3b, State: Running, Role: LEADER
I20260807 19:34:29.111804 24445 consensus_queue.cc:237] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } }
I20260807 19:34:29.117238 23915 catalog_manager.cc:5495] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b reported cstate change: term changed from 0 to 1, leader changed from <none> to eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129). New cstate: current_term: 1 leader_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } health_report { overall_health: UNKNOWN } } }
I20260807 19:34:29.165243 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.159064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50908 (local address 127.8.216.129:38695)
0807 19:34:29.159317 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.159322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.159341 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:34:29.159470 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.159475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.159558 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.159684 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:34:29.160116 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.160928 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.162625 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.163058 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.163216 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.163873 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:34:29.164568 (+ 695us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:34:29.164687 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.164816 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.164929 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 19:34:29.174233 24220 raft_consensus.cc:1261] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Refusing update from remote peer eb035a92774e4370bc69c57c5b3cbf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:29.174228 24366 raft_consensus.cc:1261] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Refusing update from remote peer eb035a92774e4370bc69c57c5b3cbf3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:29.175386 24452 consensus_queue.cc:1035] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:29.175815 24445 consensus_queue.cc:1035] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:29.205117 23914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:29.205435 23914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:29.209589 23914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Sending DeleteTablet for 3 replicas of tablet 243a7ca423b945eaaf7e48dd8e4682ba
I20260807 19:34:29.211040 24198 tablet_service.cc:1514] Processing DeleteTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:29 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:49202
I20260807 19:34:29.211299 24053 tablet_service.cc:1514] Processing DeleteTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:29 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50902
I20260807 19:34:29.211426 24344 tablet_service.cc:1514] Processing DeleteTablet for tablet 243a7ca423b945eaaf7e48dd8e4682ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:29 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:55558
I20260807 19:34:29.212435 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23980
I20260807 19:34:29.224843 24467 tablet_replica.cc:330] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: stopping tablet replica
I20260807 19:34:29.225687 24467 raft_consensus.cc:2229] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:29.228878 24467 raft_consensus.cc:2258] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:29.232005 24467 ts_tablet_manager.cc:1905] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:29.233397 24466 tablet_replica.cc:330] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: stopping tablet replica
I20260807 19:34:29.234097 24466 raft_consensus.cc:2229] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:34:29.236531 24465 tablet_replica.cc:330] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: stopping tablet replica
I20260807 19:34:29.237026 24465 raft_consensus.cc:2229] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:29.237462 24466 raft_consensus.cc:2258] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:29.237640 24465 raft_consensus.cc:2258] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:29.240221 24466 ts_tablet_manager.cc:1905] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:34:29 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:34:29.245993 24467 ts_tablet_manager.cc:1918] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:29.247494 24465 ts_tablet_manager.cc:1905] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:29.252012 24467 log.cc:1196] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/243a7ca423b945eaaf7e48dd8e4682ba
I20260807 19:34:29.252779 24467 ts_tablet_manager.cc:1939] T 243a7ca423b945eaaf7e48dd8e4682ba P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting consensus metadata
I20260807 19:34:29.253384 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.213719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.8.216.129:38695)
0807 19:34:29.213934 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.213941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.228990 (+ 15049us) server_negotiation.cc:407] Connection header received
0807 19:34:29.229591 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.229598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.229690 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.229777 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:29.232217 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.233521 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.235756 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.236615 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.236735 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.243277 (+ 6542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:29.243300 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:29.243321 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:29.243379 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:29.248797 (+ 5418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.249989 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.250001 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.250005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.250273 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.250908 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.250913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.250917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.251443 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:29.251884 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.252620 (+ 736us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.252924 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:34:29.255278 24120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:29.257017 24465 ts_tablet_manager.cc:1918] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:29.257301 24465 log.cc:1196] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/243a7ca423b945eaaf7e48dd8e4682ba
I20260807 19:34:29.257306 23901 catalog_manager.cc:4841] TS 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): tablet 243a7ca423b945eaaf7e48dd8e4682ba (table test-table [id=e4be42953def4914b7a702777ba0f6dd]) successfully deleted
I20260807 19:34:29.257967 24465 ts_tablet_manager.cc:1939] T 243a7ca423b945eaaf7e48dd8e4682ba P 06833fae4c304fd091d5e90d3d214bd5: Deleting consensus metadata
I20260807 19:34:29.260089 23900 catalog_manager.cc:4841] TS 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): tablet 243a7ca423b945eaaf7e48dd8e4682ba (table test-table [id=e4be42953def4914b7a702777ba0f6dd]) successfully deleted
I20260807 19:34:29.364656 24466 ts_tablet_manager.cc:1918] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:29.365116 24466 log.cc:1196] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/243a7ca423b945eaaf7e48dd8e4682ba
I20260807 19:34:29.365456 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23980
I20260807 19:34:29.388543 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24131
W20260807 19:34:29.388736 23901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:38695 (error 108)
I20260807 19:34:29.390724 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.390289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50918)
0807 19:34:29.390480 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.390581 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:34:29.390993 23901 catalog_manager.cc:4584] TS eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 243a7ca423b945eaaf7e48dd8e4682ba: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Aug 07 19:34:29 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:34:29.411520 24436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.389503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.8.216.130:45135)
0807 19:34:29.389696 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.389703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.389775 (+ 72us) server_negotiation.cc:407] Connection header received
0807 19:34:29.390015 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.390025 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.390095 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.390218 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:29.391795 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.392677 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.394067 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.394612 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.394762 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.404594 (+ 9832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:29.404617 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:29.404633 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:29.404681 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:29.407991 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.409051 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.409061 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.409064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.409212 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.409670 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.409675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.409685 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.410134 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:29.410521 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.410848 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.411112 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:34:29.412431 24263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:29.444523 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.444028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50920)
0807 19:34:29.444272 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.444374 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:34:29.445652 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.445282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50922)
0807 19:34:29.445434 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.445518 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 19:34:29.506743 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.506102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50924)
0807 19:34:29.506366 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.506492 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:34:29.507822 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24131
I20260807 19:34:29.508090 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.507619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50932)
0807 19:34:29.507811 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.507911 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 19:34:29.529700 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24276
Aug 07 19:34:29 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:34:29.545224 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.530528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.8.216.131:34715)
0807 19:34:29.530711 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:29.530716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:29.530739 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:34:29.530914 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:29.530918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:29.530988 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:29.531079 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:29.532462 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.533176 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.534470 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:29.534868 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:29.534968 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:29.539487 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:29.539506 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:29.539523 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:29.539564 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:29.542514 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.543286 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.543294 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.543296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.543474 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:29.543815 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:29.543818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:29.543821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:29.544270 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:29.544532 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:29.544759 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 19:34:29.544936 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:34:29.545920 24412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:29.589502 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.588777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50936)
0807 19:34:29.589077 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.589203 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 19:34:29.590909 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.590369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50944)
0807 19:34:29.590644 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.590752 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260807 19:34:29.638792 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24276
I20260807 19:34:29.661173 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 23881
I20260807 19:34:29.662580 23947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:29.784401 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:29.783586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50946)
0807 19:34:29.783894 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:29.784036 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 19:34:29.784696 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 23881
I20260807 19:34:29.812741 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--webserver_interface=127.8.216.190
--webserver_port=43357
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:30.185922 24475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:30.186300 24475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:30.186483 24475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:30.194697 24475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:30.194846 24475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:30.194954 24475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:30.195055 24475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:30.195151 24475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:30.206274 24475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=43357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:30.207810 24475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:30.209345 24475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:30.239053 24481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:30.239027 24484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:30.238988 24482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:30.240214 24475 server_base.cc:1034] running on GCE node
I20260807 19:34:30.241360 24475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:30.243257 24475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:30.244539 24475 hybrid_clock.cc:648] HybridClock initialized: now 1786131270244501 us; error 91 us; skew 500 ppm
I20260807 19:34:30.245172 24475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:30 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131270, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:30.249361 24475 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:30.251405 24475 webserver.cc:460] Webserver started at http://127.8.216.190:43357/ using document root <none> and password file <none>
I20260807 19:34:30.252441 24475 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:30.252635 24475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:30.259955 24475 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:34:30.265069 24493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:30.267059 24475 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:34:30.267284 24475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "073f78f2402a4bbbba28d53013f70faf"
format_stamp: "Formatted at 2026-08-07 19:34:26 on dist-test-slave-8w9v"
server_key: "962d3b2b28712c4d4386490abe67badc"
server_key_iv: "ea617d8467eab5db44abbcbf15d63c88"
server_key_version: "encryptionkey@0"
I20260807 19:34:30.268603 24475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:30.289561 24475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:30.294610 24475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:30.294919 24475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:30.317198 24475 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36933
I20260807 19:34:30.317198 24546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36933 every 8 connection(s)
I20260807 19:34:30.319024 24475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:30.326270 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24475
I20260807 19:34:30.327260 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:38695
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=33197
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:30.329276 24547 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:34:30.333207 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap starting.
I20260807 19:34:30.355731 24547 log.cc:824] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:30.370035 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:34:30.370663 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap complete.
I20260807 19:34:30.381417 24547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:30.382388 24547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Initialized, Role: FOLLOWER
I20260807 19:34:30.383076 24547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:30.383355 24547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:30.383536 24547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:30.383733 24547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:34:30.388324 24547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:30.389034 24547 leader_election.cc:302] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 073f78f2402a4bbbba28d53013f70faf; no voters:
I20260807 19:34:30.389745 24547 leader_election.cc:288] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:34:30.389906 24552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:34:30.391290 24552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 LEADER]: Becoming Leader. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Running, Role: LEADER
I20260807 19:34:30.392040 24552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:30.392617 24547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:30.394760 24555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073f78f2402a4bbbba28d53013f70faf. Latest consensus state: current_term: 2 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:30.395190 24555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:30.394966 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:30.395385 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:30.401572 24561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:30.412501 24561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:30.414357 24561 catalog_manager.cc:1259] Loaded cluster ID: 6788afe0604649efb9a89a553bd04531
I20260807 19:34:30.414635 24561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:30.418736 24561 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 19:34:30.437491 24561 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 19:34:30.457837 24561 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:30.459851 24561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Loaded TSK: 0
I20260807 19:34:30.461757 24561 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:34:30.737993 24550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:30.738407 24550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:30.738571 24550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:30.746490 24550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:30.746665 24550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:30.746863 24550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:30.757552 24550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:38695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=33197
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:30.759070 24550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:30.760507 24550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:30.777242 24578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:30.777843 24577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:30.777359 24581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:30.778353 24550 server_base.cc:1034] running on GCE node
I20260807 19:34:30.779124 24550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:30.780355 24550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:30.781610 24550 hybrid_clock.cc:648] HybridClock initialized: now 1786131270781545 us; error 62 us; skew 500 ppm
I20260807 19:34:30.781973 24550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:30 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131270, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:30.785619 24550 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:30.787485 24550 webserver.cc:460] Webserver started at http://127.8.216.129:33197/ using document root <none> and password file <none>
I20260807 19:34:30.788331 24550 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:30.788511 24550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:30.794804 24550 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:34:30.798218 24588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:30.800307 24550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:30.800526 24550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
format_stamp: "Formatted at 2026-08-07 19:34:27 on dist-test-slave-8w9v"
server_key: "725e44ba96a5c374186eac3fc9a6dd93"
server_key_iv: "7b5e171b9629bc11b7765e2ee69b9f39"
server_key_version: "encryptionkey@0"
I20260807 19:34:30.801728 24550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:30.826018 24550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:30.831277 24550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:30.831637 24550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:30.832944 24550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:30.835152 24596 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 19:34:30.841248 24596 ts_tablet_manager.cc:1852] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 19:34:30.842393 24596 ts_tablet_manager.cc:1905] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:30.850725 24596 ts_tablet_manager.cc:1918] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:30.850999 24596 log.cc:1196] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/243a7ca423b945eaaf7e48dd8e4682ba
I20260807 19:34:30.851514 24596 ts_tablet_manager.cc:1939] T 243a7ca423b945eaaf7e48dd8e4682ba P eb035a92774e4370bc69c57c5b3cbf3b: Deleting consensus metadata
I20260807 19:34:30.852011 24550 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 19:34:30.852308 24550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260807 19:34:30.852504 24550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:30.852610 24550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:30.893373 24550 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38695
I20260807 19:34:30.893481 24712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38695 every 8 connection(s)
I20260807 19:34:30.895156 24550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:30.901587 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24550
I20260807 19:34:30.902448 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:45135
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=35803
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:30 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131270, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:30.927215 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:30.898504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:45369 (local address 127.8.216.190:36933)
0807 19:34:30.899078 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:30.899092 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:30.899887 (+ 795us) server_negotiation.cc:407] Connection header received
0807 19:34:30.903263 (+ 3376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:30.903285 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:30.903811 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:30.904282 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:30.906172 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:30.908333 (+ 2161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:30.910181 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:30.911334 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:30.911563 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:30.916656 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:30.916666 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:30.916691 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:30.916746 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:30.920957 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:30.922529 (+ 1572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:30.922539 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:30.922546 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:30.922789 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:30.923584 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:30.923590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:30.923592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:30.924413 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:30.924605 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:30.925815 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0807 19:34:30.926194 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":262,"threads_started":1}
I20260807 19:34:30.927680 24717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:30.898882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:45369)
0807 19:34:30.899698 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:30.899737 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:30.902420 (+ 2683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:30.904538 (+ 2118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:30.904547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:30.905171 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:30.905883 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:30.905916 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:30.908580 (+ 2664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:30.908585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:30.909983 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:30.910000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:30.911643 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:30.911650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:30.911916 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:30.912988 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:30.913038 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:30.916272 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:30.921242 (+ 4970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:30.921258 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:30.921280 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:30.922241 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:30.922967 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:30.922976 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:30.922988 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:30.923381 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:30.924603 (+ 1222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:30.924617 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:30.925186 (+ 569us) client_negotiation.cc:769] Sending connection context
0807 19:34:30.925540 (+ 354us) client_negotiation.cc:240] Negotiation successful
0807 19:34:30.926218 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":310,"threads_started":1}
I20260807 19:34:30.931648 24714 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:30.932099 24714 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:30.933073 24714 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:30.936470 24510 ts_manager.cc:194] Registered new tserver with Master: eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695)
I20260807 19:34:30.938961 24510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:45369
W20260807 19:34:31.282153 24718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:31.282513 24718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:31.282617 24718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:31.290483 24718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:31.290619 24718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:31.290740 24718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:31.301311 24718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:45135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=35803
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:31.302842 24718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:31.304364 24718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:31.319777 24723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:31.319880 24724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:31.320508 24726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:31.321151 24718 server_base.cc:1034] running on GCE node
I20260807 19:34:31.321849 24718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:31.322997 24718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:31.324292 24718 hybrid_clock.cc:648] HybridClock initialized: now 1786131271324220 us; error 52 us; skew 500 ppm
I20260807 19:34:31.324769 24718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:31 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:31.330159 24718 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:31.332059 24718 webserver.cc:460] Webserver started at http://127.8.216.130:35803/ using document root <none> and password file <none>
I20260807 19:34:31.332916 24718 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:31.333091 24718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:31.339584 24718 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 19:34:31.342829 24734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:31.344556 24718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:34:31.344779 24718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "06833fae4c304fd091d5e90d3d214bd5"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "27eaae018a3115e11b8ae30b8ac0062f"
server_key_iv: "4f81ed7c5baeec504c20c20d184c5b24"
server_key_version: "encryptionkey@0"
I20260807 19:34:31.345927 24718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:31.367345 24718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:31.372594 24718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:31.372936 24718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:31.374200 24718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:31.375783 24718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:31.375929 24718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:31.376089 24718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:31.376206 24718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 19:34:31.414285 24718 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:45135
I20260807 19:34:31.414345 24853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:45135 every 8 connection(s)
I20260807 19:34:31.415849 24718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:31.420329 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24718
I20260807 19:34:31.421137 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:34715
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=43957
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:34:31 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:31.443907 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.418923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:45139 (local address 127.8.216.190:36933)
0807 19:34:31.419141 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:31.419146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:31.420665 (+ 1519us) server_negotiation.cc:407] Connection header received
0807 19:34:31.423979 (+ 3314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:31.423985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:31.424090 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:31.424272 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:31.426826 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.427601 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.430362 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.430834 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.430948 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.435610 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:31.435621 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:31.435630 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:31.435688 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:31.439907 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.440859 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.440868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.440875 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.441095 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.441622 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.441631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.441637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.442096 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:31.442290 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:31.443086 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 19:34:31.443396 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:34:31.444592 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.419421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:45139)
0807 19:34:31.420435 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:31.420498 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:31.423505 (+ 3007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:31.424612 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:31.424628 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:31.425583 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:31.426545 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:31.426578 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.427814 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.427823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:31.430129 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:31.430151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.431104 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.431114 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:31.431548 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.432539 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:31.432575 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:31.435301 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:31.440102 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:31.440110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:31.440177 (+ 67us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:31.440690 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:31.441233 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:31.441236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:31.441239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:31.441473 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:31.442234 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:31.442241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:31.442621 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 19:34:31.442874 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 19:34:31.443305 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":372,"threads_started":1}
I20260807 19:34:31.446295 24854 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:31.446671 24854 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:31.447540 24854 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:31.449273 24510 ts_manager.cc:194] Registered new tserver with Master: 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
I20260807 19:34:31.450340 24510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:45139
W20260807 19:34:31.811357 24857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:31.811717 24857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:31.811946 24857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:31.819916 24857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:31.820065 24857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:31.820281 24857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:31.830786 24857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:34715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=43957
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:31.832265 24857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:31.833699 24857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:31.848902 24864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:31.848876 24867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:31.848984 24865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:31.850144 24857 server_base.cc:1034] running on GCE node
I20260807 19:34:31.850934 24857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:31.852262 24857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:31.853546 24857 hybrid_clock.cc:648] HybridClock initialized: now 1786131271853492 us; error 61 us; skew 500 ppm
I20260807 19:34:31.853914 24857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:31 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:31.857604 24857 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:31.859671 24857 webserver.cc:460] Webserver started at http://127.8.216.131:43957/ using document root <none> and password file <none>
I20260807 19:34:31.860647 24857 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:31.860785 24857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:31.867162 24857 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 19:34:31.870692 24875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:31.872516 24857 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:31.872689 24857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "c10b17eb8eaf813bd8278e5c24538eab"
server_key_iv: "f7e1327e4719061bbd80cf6c3c42b417"
server_key_version: "encryptionkey@0"
I20260807 19:34:31.873860 24857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:31.892493 24857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:31.898902 24857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:31.899250 24857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:31.900590 24857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:31.902060 24857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:31.902159 24857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:31.902243 24857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:31.902312 24857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:31.940043 24857 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:34715
I20260807 19:34:31.940225 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:34715 every 8 connection(s)
I20260807 19:34:31.941748 24857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:31.942966 24714 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
I20260807 19:34:31.948945 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 24857
Aug 07 19:34:31 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:31.965457 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.949990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.8.216.190:36933)
0807 19:34:31.950582 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:31.950588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:31.951147 (+ 559us) server_negotiation.cc:407] Connection header received
0807 19:34:31.951330 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:31.951334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:31.951481 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:31.951573 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:31.953424 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.954225 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.955888 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.956423 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.956561 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.959402 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:31.959406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:31.959409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:31.959452 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:31.962427 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.963193 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.963200 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.963203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.963361 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.963775 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.963782 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.963790 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.964068 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:31.964447 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:31.964754 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:34:31.965012 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":283,"threads_started":1}
I20260807 19:34:31.966095 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.944862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:58809 (local address 127.8.216.190:36933)
0807 19:34:31.945436 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:31.945443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:31.946354 (+ 911us) server_negotiation.cc:407] Connection header received
0807 19:34:31.948380 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:31.948386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:31.948481 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:31.948624 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:31.951033 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.952363 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.954070 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.954554 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.954697 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.958661 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:31.958667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:31.958672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:31.958719 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:31.962190 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.963010 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.963018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.963022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.963229 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:31.963701 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:31.963708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:31.963712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:31.964015 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:31.964203 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:31.965002 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 19:34:31.965224 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":303,"threads_started":1}
I20260807 19:34:31.967088 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.945393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:58809)
0807 19:34:31.946189 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:31.946234 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:31.948025 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:31.948915 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:31.948925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:31.949698 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:31.950808 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:31.950840 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.952572 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.952576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:31.953881 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:31.953894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.954814 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.954822 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:31.955083 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.955829 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:31.955853 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:31.958424 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:31.962358 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:31.962368 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:31.962388 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:31.962875 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:31.963345 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:31.963351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:31.963355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:31.963580 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:31.964177 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:31.964186 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:31.964632 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 19:34:31.964916 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 19:34:31.965538 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":284,"threads_started":1}
I20260807 19:34:31.968719 24998 heartbeater.cc:347] Connected to a master server at 127.8.216.190:36933
I20260807 19:34:31.969148 24998 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:31.969966 24998 heartbeater.cc:511] Master 127.8.216.190:36933 requested a full tablet report, sending...
I20260807 19:34:31.971586 24509 ts_manager.cc:194] Registered new tserver with Master: 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715)
I20260807 19:34:31.972653 24509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:58809
I20260807 19:34:31.982347 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:34:31.989564 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:31.983343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.8.216.190:36933)
0807 19:34:31.983526 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:31.983530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:31.984062 (+ 532us) server_negotiation.cc:407] Connection header received
0807 19:34:31.984339 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:31.984343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:31.984523 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:31.984656 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:34:31.985092 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.985840 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.987365 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:31.987801 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:31.987901 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:31.988465 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:34:31.989038 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:34:31.989153 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:31.989270 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 19:34:31.989343 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:34:31.990581 24509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:34:31.999218 24509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:34:32.035549 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.013343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.8.216.130:45135)
0807 19:34:32.023544 (+ 10201us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.023560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.025204 (+ 1644us) server_negotiation.cc:407] Connection header received
0807 19:34:32.025691 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.025700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.025970 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.026155 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:32.027927 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.029506 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.032485 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.034092 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.034251 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.034478 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.034951 (+ 473us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.035155 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9915,"thread_start_us":10229,"threads_started":1}
I20260807 19:34:32.035742 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.013140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.8.216.131:34715)
0807 19:34:32.024411 (+ 11271us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.024420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.025202 (+ 782us) server_negotiation.cc:407] Connection header received
0807 19:34:32.025691 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.025700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.025978 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.026155 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:32.029185 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.030891 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.033269 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.034475 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.034612 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.034845 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.035165 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.035341 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11087,"thread_start_us":10890,"threads_started":1}
I20260807 19:34:32.035766 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.012948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:34715 (local address 127.0.0.1:55582)
0807 19:34:32.025002 (+ 12054us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.025071 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:32.025400 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:32.026540 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:32.026549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:32.026593 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:32.027664 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.027683 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.031090 (+ 3407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.031096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.033095 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.033108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.034730 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.034737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.034922 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.034971 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 19:34:32.035067 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:34:32.035293 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11728,"mutex_wait_us":490,"spinlock_wait_cycles":10112,"thread_start_us":11533,"threads_started":1}
I20260807 19:34:32.037462 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.012960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45135 (local address 127.0.0.1:49220)
0807 19:34:32.025002 (+ 12042us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.025071 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:32.025400 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:32.026540 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:32.026549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:32.026593 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:32.027180 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.027197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.029672 (+ 2475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.029679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.032288 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.032307 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.034369 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.034377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.034614 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.034683 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 19:34:32.036053 (+ 1370us) client_negotiation.cc:240] Negotiation successful
0807 19:34:32.036253 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11845,"mutex_wait_us":242,"spinlock_wait_cycles":20480,"thread_start_us":11459,"threads_started":1}
I20260807 19:34:32.038053 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.027108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50954)
0807 19:34:32.027904 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.027927 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:32.028050 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:32.028808 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:32.028811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:32.028834 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:32.029228 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.029242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.030683 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.030690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.033113 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.033130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.037114 (+ 3984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.037121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.037321 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.037357 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:34:32.037483 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 19:34:32.037651 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":28,"thread_start_us":393,"threads_started":1}
I20260807 19:34:32.038874 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.026101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.8.216.129:38695)
0807 19:34:32.028177 (+ 2076us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.028189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.028222 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:34:32.028338 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.028348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.028620 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.028814 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:32.029474 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.030535 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.033322 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.034987 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.035112 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.035326 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.038403 (+ 3077us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.038611 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1858,"thread_start_us":1713,"threads_started":1}
I20260807 19:34:32.040247 24926 tablet_service.cc:1467] Processing CreateTablet for tablet b777056c044f4afc81b93a9419c29e6a (DEFAULT_TABLE table=test-table [id=b445bb3859454b069ba46b52c8ce5b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:32.040978 24641 tablet_service.cc:1467] Processing CreateTablet for tablet b777056c044f4afc81b93a9419c29e6a (DEFAULT_TABLE table=test-table [id=b445bb3859454b069ba46b52c8ce5b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:32.041774 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b777056c044f4afc81b93a9419c29e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:32.042225 24641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b777056c044f4afc81b93a9419c29e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:32.043742 24784 tablet_service.cc:1467] Processing CreateTablet for tablet b777056c044f4afc81b93a9419c29e6a (DEFAULT_TABLE table=test-table [id=b445bb3859454b069ba46b52c8ce5b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:32.045807 24784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b777056c044f4afc81b93a9419c29e6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:32.050405 25018 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Bootstrap starting.
I20260807 19:34:32.054481 25018 tablet_bootstrap.cc:654] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:32.055861 25018 log.cc:824] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:32.058735 25020 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Bootstrap starting.
I20260807 19:34:32.059439 25018 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: No bootstrap required, opened a new log
I20260807 19:34:32.059736 25018 ts_tablet_manager.cc:1397] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:34:32.062866 25021 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Bootstrap starting.
I20260807 19:34:32.063127 25020 tablet_bootstrap.cc:654] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:32.064841 25020 log.cc:824] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:32.066955 25021 tablet_bootstrap.cc:654] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:32.068375 25021 log.cc:824] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:32.071395 25020 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: No bootstrap required, opened a new log
I20260807 19:34:32.071635 25021 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: No bootstrap required, opened a new log
I20260807 19:34:32.071755 25020 ts_tablet_manager.cc:1397] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260807 19:34:32.071512 25018 raft_consensus.cc:348] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.071899 25021 ts_tablet_manager.cc:1397] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:34:32.072078 25018 raft_consensus.cc:374] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:32.072213 25018 raft_consensus.cc:729] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb035a92774e4370bc69c57c5b3cbf3b, State: Initialized, Role: FOLLOWER
I20260807 19:34:32.072942 25018 consensus_queue.cc:260] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.074976 25018 ts_tablet_manager.cc:1428] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
I20260807 19:34:32.083920 25020 raft_consensus.cc:348] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.083920 25021 raft_consensus.cc:348] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.084535 25020 raft_consensus.cc:374] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:32.084534 25021 raft_consensus.cc:374] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:32.084662 25021 raft_consensus.cc:729] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06833fae4c304fd091d5e90d3d214bd5, State: Initialized, Role: FOLLOWER
I20260807 19:34:32.084662 25020 raft_consensus.cc:729] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9e42b690044ea9a6b57f7455f8c41e, State: Initialized, Role: FOLLOWER
I20260807 19:34:32.085397 25020 consensus_queue.cc:260] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.085397 25021 consensus_queue.cc:260] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.086720 24998 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
I20260807 19:34:32.087219 24854 heartbeater.cc:503] Master 127.8.216.190:36933 was elected leader, sending a full tablet report...
I20260807 19:34:32.087636 25020 ts_tablet_manager.cc:1428] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Time spent starting tablet: real 0.016s user 0.011s sys 0.006s
I20260807 19:34:32.087770 25021 ts_tablet_manager.cc:1428] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
W20260807 19:34:32.148162 24715 tablet.cc:2369] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:34:32.168526 24855 tablet.cc:2369] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:34:32.193593 24999 tablet.cc:2369] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:32.293480 25026 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:32.293852 25026 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.295893 25026 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135), eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695)
I20260807 19:34:32.301885 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.296231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.8.216.131:53639)
0807 19:34:32.296561 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.296583 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:32.296720 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:32.297176 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:32.297180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:32.297211 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:32.297413 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.297421 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.298472 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.298479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.299822 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.299838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.301061 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.301068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.301296 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.301330 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:34:32.301442 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:34:32.301583 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 19:34:32.301892 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.296642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:53639 (local address 127.8.216.129:38695)
0807 19:34:32.296884 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.296889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.296904 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:34:32.296969 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.296973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.297039 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.297150 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:32.297569 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.298334 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.299995 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.300902 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.301075 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.301153 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.301471 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.301621 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:34:32.302819 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.296461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48607 (local address 127.8.216.130:45135)
0807 19:34:32.296743 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.296750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.297454 (+ 704us) server_negotiation.cc:407] Connection header received
0807 19:34:32.297666 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.297670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.297734 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.297838 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:32.298400 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.299083 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.300889 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.301802 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.301990 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.302063 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.302417 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.302556 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:34:32.303025 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.296284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45135 (local address 127.8.216.131:48607)
0807 19:34:32.297325 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.297341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:32.297504 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:32.297849 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:32.297854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:32.297886 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:32.298218 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.298225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.299238 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.299243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.300724 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:32.300737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.302004 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.302011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:32.302213 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.302261 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 19:34:32.302381 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 19:34:32.302515 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":663,"threads_started":1}
I20260807 19:34:32.303536 24662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b777056c044f4afc81b93a9419c29e6a" candidate_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" is_pre_election: true
I20260807 19:34:32.304195 24662 raft_consensus.cc:2454] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9e42b690044ea9a6b57f7455f8c41e in term 0.
I20260807 19:34:32.304198 24806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b777056c044f4afc81b93a9419c29e6a" candidate_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06833fae4c304fd091d5e90d3d214bd5" is_pre_election: true
I20260807 19:34:32.304731 24806 raft_consensus.cc:2454] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a9e42b690044ea9a6b57f7455f8c41e in term 0.
I20260807 19:34:32.304996 24878 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e42b690044ea9a6b57f7455f8c41e, eb035a92774e4370bc69c57c5b3cbf3b; no voters:
I20260807 19:34:32.305516 25026 raft_consensus.cc:2781] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:34:32.305733 25026 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:34:32.305907 25026 raft_consensus.cc:3037] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:32.308463 25026 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.309137 25026 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [CANDIDATE]: Term 1 election: Requested vote from peers 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135), eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695)
I20260807 19:34:32.309770 24806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b777056c044f4afc81b93a9419c29e6a" candidate_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06833fae4c304fd091d5e90d3d214bd5"
I20260807 19:34:32.309825 24662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b777056c044f4afc81b93a9419c29e6a" candidate_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
I20260807 19:34:32.310006 24806 raft_consensus.cc:3037] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:32.310065 24662 raft_consensus.cc:3037] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:32.312866 24662 raft_consensus.cc:2454] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9e42b690044ea9a6b57f7455f8c41e in term 1.
I20260807 19:34:32.312902 24806 raft_consensus.cc:2454] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a9e42b690044ea9a6b57f7455f8c41e in term 1.
I20260807 19:34:32.313340 24878 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a9e42b690044ea9a6b57f7455f8c41e, eb035a92774e4370bc69c57c5b3cbf3b; no voters:
I20260807 19:34:32.313688 25026 raft_consensus.cc:2781] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:32.314199 25026 raft_consensus.cc:686] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 LEADER]: Becoming Leader. State: Replica: 1a9e42b690044ea9a6b57f7455f8c41e, State: Running, Role: LEADER
I20260807 19:34:32.314783 25026 consensus_queue.cc:237] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:32.320648 24509 catalog_manager.cc:5495] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131). New cstate: current_term: 1 leader_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } health_report { overall_health: UNKNOWN } } }
I20260807 19:34:32.397661 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.391760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55588 (local address 127.8.216.131:34715)
0807 19:34:32.391938 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.391943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.391961 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:34:32.392116 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.392160 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.392261 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.392401 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:34:32.392772 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.393622 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.395146 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.395562 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.395668 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.396445 (+ 777us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:34:32.397036 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:34:32.397171 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.397360 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.397441 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:34:32.406783 24806 raft_consensus.cc:1261] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Refusing update from remote peer 1a9e42b690044ea9a6b57f7455f8c41e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:32.406847 24662 raft_consensus.cc:1261] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Refusing update from remote peer 1a9e42b690044ea9a6b57f7455f8c41e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:32.407919 25028 consensus_queue.cc:1035] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:32.408419 25026 consensus_queue.cc:1035] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [LEADER]: Connected to new peer: Peer: permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:32.421391 25031 mvcc.cc:205] Tried to move back new op lower bound from 7315993691766874112 to 7315993691405893632. Current Snapshot: MvccSnapshot[applied={T|T < 7315993691766874112}]
I20260807 19:34:32.430047 25032 mvcc.cc:205] Tried to move back new op lower bound from 7315993691766874112 to 7315993691405893632. Current Snapshot: MvccSnapshot[applied={T|T < 7315993691766874112}]
I20260807 19:34:32.447405 24509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:32.447713 24509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:32.452191 24509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Sending DeleteTablet for 3 replicas of tablet b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:32.453711 24784 tablet_service.cc:1514] Processing DeleteTablet for tablet b777056c044f4afc81b93a9419c29e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:32 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:49220
I20260807 19:34:32.453697 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet b777056c044f4afc81b93a9419c29e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:32 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:55582
I20260807 19:34:32.453759 24641 tablet_service.cc:1514] Processing DeleteTablet for tablet b777056c044f4afc81b93a9419c29e6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:32 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:50954
I20260807 19:34:32.454082 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24550
I20260807 19:34:32.469353 25045 tablet_replica.cc:330] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: stopping tablet replica
I20260807 19:34:32.469988 25045 raft_consensus.cc:2229] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:32.470638 25045 raft_consensus.cc:2258] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:32.472960 25044 tablet_replica.cc:330] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: stopping tablet replica
I20260807 19:34:32.473965 25045 ts_tablet_manager.cc:1905] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:32.473460 25044 raft_consensus.cc:2229] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:34:32.476306 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.455027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50968 (local address 127.8.216.129:38695)
0807 19:34:32.455204 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.455210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.455346 (+ 136us) server_negotiation.cc:407] Connection header received
0807 19:34:32.455568 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.455574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.455666 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.455774 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:32.458210 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.459444 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.462007 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.462775 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.462917 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.467572 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:32.467595 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:32.467617 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:32.467666 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:32.472399 (+ 4733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.473635 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.473647 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.473651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.473866 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.474317 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.474323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.474326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.474845 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:32.475220 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.475567 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.475826 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:34:32.476533 25043 tablet_replica.cc:330] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: stopping tablet replica
I20260807 19:34:32.477082 24707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:32.477439 25044 raft_consensus.cc:2258] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:32.478199 25043 raft_consensus.cc:2229] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:32.479094 25043 raft_consensus.cc:2258] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:32.482187 25043 ts_tablet_manager.cc:1905] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:32.482244 25044 ts_tablet_manager.cc:1905] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:32.490362 25043 ts_tablet_manager.cc:1918] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:32.490597 25043 log.cc:1196] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:32.491240 25043 ts_tablet_manager.cc:1939] T b777056c044f4afc81b93a9419c29e6a P 06833fae4c304fd091d5e90d3d214bd5: Deleting consensus metadata
I20260807 19:34:32.491550 25044 ts_tablet_manager.cc:1918] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:32.491852 25044 log.cc:1196] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:32.492769 25044 ts_tablet_manager.cc:1939] T b777056c044f4afc81b93a9419c29e6a P 1a9e42b690044ea9a6b57f7455f8c41e: Deleting consensus metadata
I20260807 19:34:32.493635 24495 catalog_manager.cc:4841] TS 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): tablet b777056c044f4afc81b93a9419c29e6a (table test-table [id=b445bb3859454b069ba46b52c8ce5b14]) successfully deleted
I20260807 19:34:32.495000 24496 catalog_manager.cc:4841] TS 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): tablet b777056c044f4afc81b93a9419c29e6a (table test-table [id=b445bb3859454b069ba46b52c8ce5b14]) successfully deleted
I20260807 19:34:32.580773 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24550
I20260807 19:34:32.604264 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24718
W20260807 19:34:32.604357 24496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:38695 (error 108)
I20260807 19:34:32.606673 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.605856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50970)
0807 19:34:32.606309 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.606447 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"threads_started":1}
W20260807 19:34:32.607035 24496 catalog_manager.cc:4584] TS eb035a92774e4370bc69c57c5b3cbf3b (127.8.216.129:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b777056c044f4afc81b93a9419c29e6a: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
I20260807 19:34:32.622352 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.605189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.8.216.130:45135)
0807 19:34:32.605391 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.605397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.606055 (+ 658us) server_negotiation.cc:407] Connection header received
0807 19:34:32.606154 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.606160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.606272 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.606397 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:32.608332 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.609085 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.611179 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.611619 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.611723 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.615574 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:32.615594 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:32.615612 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:32.615653 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:32.619047 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.620058 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.620067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.620069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.620271 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.620711 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.620714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.620716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.621135 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:32.621369 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.621796 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.622016 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:34:32.623068 24849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:32.660861 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.660281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50974)
0807 19:34:32.660570 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.660687 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 19:34:32.662117 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.661665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50976)
0807 19:34:32.661847 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.661960 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:34:32.719637 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24718
I20260807 19:34:32.725034 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.724388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50980)
0807 19:34:32.724674 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.724799 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 19:34:32.726385 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.725872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50990)
0807 19:34:32.726079 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.726185 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:34:32.744812 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24857
I20260807 19:34:32.762744 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.745868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.8.216.131:34715)
0807 19:34:32.746065 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:32.746070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:32.746089 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:34:32.746219 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:32.746223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:32.746311 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:32.746416 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:32.748428 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.749278 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.751225 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:32.751663 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:32.751755 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:32.755569 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:32.755593 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:32.755610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:32.755659 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:32.759151 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.760294 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.760303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.760306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.760529 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:32.761040 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:32.761044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:32.761046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:32.761455 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:32.761701 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:32.762139 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 19:34:32.762330 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:34:32.763700 24993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:32.807168 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.806653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:50992)
0807 19:34:32.806889 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.806993 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:34:32.808426 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:32.807921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:51004)
0807 19:34:32.808176 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:32.808259 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 19:34:32.896214 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24857
I20260807 19:34:32.922024 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 24475
I20260807 19:34:32.923233 24542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:33.044386 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:33.043643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:38695 (local address 127.0.0.1:51006)
0807 19:34:33.043960 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:33.044114 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 19:34:33.044519 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 24475
I20260807 19:34:33.073671 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--webserver_interface=127.8.216.190
--webserver_port=43357
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:33.451051 25052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:33.451452 25052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:33.451632 25052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:33.459884 25052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:33.460067 25052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:33.460197 25052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:33.460312 25052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:33.460388 25052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:33.471638 25052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=43357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:33.473124 25052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:33.474607 25052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:33.487681 25057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:33.487718 25058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:33.487777 25061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:33.489435 25052 server_base.cc:1034] running on GCE node
I20260807 19:34:33.490365 25052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:33.492050 25052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:33.493353 25052 hybrid_clock.cc:648] HybridClock initialized: now 1786131273493270 us; error 67 us; skew 500 ppm
I20260807 19:34:33.493765 25052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:33 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131273, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:33.503136 25052 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:33.505234 25052 webserver.cc:460] Webserver started at http://127.8.216.190:43357/ using document root <none> and password file <none>
I20260807 19:34:33.506057 25052 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:33.506227 25052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:33.512455 25052 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:33.518376 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:33.519860 25052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:34:33.520083 25052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "073f78f2402a4bbbba28d53013f70faf"
format_stamp: "Formatted at 2026-08-07 19:34:26 on dist-test-slave-8w9v"
server_key: "962d3b2b28712c4d4386490abe67badc"
server_key_iv: "ea617d8467eab5db44abbcbf15d63c88"
server_key_version: "encryptionkey@0"
I20260807 19:34:33.521219 25052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:33.547032 25052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:33.551571 25052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:33.551883 25052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:33.572947 25052 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36933
I20260807 19:34:33.572971 25123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36933 every 8 connection(s)
I20260807 19:34:33.574626 25052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:33.576617 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25052
I20260807 19:34:33.577687 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:38695
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=33197
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:33.585168 25124 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:34:33.590921 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap starting.
I20260807 19:34:33.613786 25124 log.cc:824] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:33.631955 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:34:33.632619 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf: Bootstrap complete.
I20260807 19:34:33.642663 25124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:33.643617 25124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Initialized, Role: FOLLOWER
I20260807 19:34:33.644407 25124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:33.644656 25124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:33.644793 25124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:33.645001 25124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:34:33.649192 25124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:33.649756 25124 leader_election.cc:302] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 073f78f2402a4bbbba28d53013f70faf; no voters:
I20260807 19:34:33.650493 25124 leader_election.cc:288] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:34:33.650651 25128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:34:33.651970 25128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [term 3 LEADER]: Becoming Leader. State: Replica: 073f78f2402a4bbbba28d53013f70faf, State: Running, Role: LEADER
I20260807 19:34:33.652727 25128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } }
I20260807 19:34:33.652951 25124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:33.655575 25129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:33.656330 25129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:33.659137 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073f78f2402a4bbbba28d53013f70faf. Latest consensus state: current_term: 3 leader_uuid: "073f78f2402a4bbbba28d53013f70faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073f78f2402a4bbbba28d53013f70faf" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36933 } } }
I20260807 19:34:33.659529 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073f78f2402a4bbbba28d53013f70faf [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:33.661666 25134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:33.672847 25134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:33.674512 25134 catalog_manager.cc:1259] Loaded cluster ID: 6788afe0604649efb9a89a553bd04531
I20260807 19:34:33.674636 25134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 19:34:33.677098 25134 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb25e6f29ad google::LogMessage::Fail() at ??:0
@ 0x7fb25e6f588a google::LogMessage::SendToLog() at ??:0
@ 0x7fb25e6f24de google::LogMessage::Flush() at ??:0
@ 0x7fb25e6f3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb274125512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb274122976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb2741aa610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb2741aa439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb26133e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb26134191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb261341799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb26131ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb2645f66db start_thread at ??:0
@ 0x7fb25d58c61f clone at ??:0
W20260807 19:34:33.991529 25126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:33.991920 25126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:33.992108 25126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:33.999930 25126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:34.000166 25126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:34.000399 25126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:34.011121 25126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:38695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=33197
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:34.012709 25126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:34.014235 25126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:34.029906 25172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:34.030000 25169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:34.030230 25170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:34.030707 25126 server_base.cc:1034] running on GCE node
I20260807 19:34:34.031528 25126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:34.032943 25126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:34.034204 25126 hybrid_clock.cc:648] HybridClock initialized: now 1786131274034155 us; error 49 us; skew 500 ppm
I20260807 19:34:34.034582 25126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:34 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131274, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:34.038178 25126 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:34.039904 25126 webserver.cc:460] Webserver started at http://127.8.216.129:33197/ using document root <none> and password file <none>
I20260807 19:34:34.040828 25126 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:34.040999 25126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:34.047113 25126 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:34:34.050770 25181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:34.052460 25126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:34:34.052700 25126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "eb035a92774e4370bc69c57c5b3cbf3b"
format_stamp: "Formatted at 2026-08-07 19:34:27 on dist-test-slave-8w9v"
server_key: "725e44ba96a5c374186eac3fc9a6dd93"
server_key_iv: "7b5e171b9629bc11b7765e2ee69b9f39"
server_key_version: "encryptionkey@0"
I20260807 19:34:34.053835 25126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:34.082031 25126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:34.087258 25126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:34.087641 25126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:34.088935 25126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:34.091490 25188 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:34:34.098244 25126 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:34:34.098488 25126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260807 19:34:34.098690 25126 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:34:34.102453 25126 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:34:34.102584 25126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260807 19:34:34.102977 25188 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Bootstrap starting.
I20260807 19:34:34.112592 25188 log.cc:824] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:34.130192 25188 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:34:34.130923 25188 tablet_bootstrap.cc:492] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Bootstrap complete.
I20260807 19:34:34.131441 25188 ts_tablet_manager.cc:1397] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20260807 19:34:34.140241 25188 raft_consensus.cc:348] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:34.141901 25188 raft_consensus.cc:729] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb035a92774e4370bc69c57c5b3cbf3b, State: Initialized, Role: FOLLOWER
I20260807 19:34:34.142799 25188 consensus_queue.cc:260] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:34.144706 25188 ts_tablet_manager.cc:1428] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b: Time spent starting tablet: real 0.013s user 0.016s sys 0.000s
I20260807 19:34:34.152094 25126 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:38695
I20260807 19:34:34.152236 25304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:38695 every 8 connection(s)
I20260807 19:34:34.153569 25126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:34.158968 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.156701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:54467)
0807 19:34:34.157539 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.157955 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":332,"threads_started":1}
W20260807 19:34:34.159641 25305 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:36933: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
I20260807 19:34:34.160673 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.160282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:53301)
0807 19:34:34.160463 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.160548 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:34:34.161623 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.161245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:43999)
0807 19:34:34.161401 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.161484 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:34:34.161912 25305 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:34:34.163235 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25126
I20260807 19:34:34.164033 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:45135
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=35803
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:34.536188 25310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:34.536504 25310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:34.536707 25310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:34.544301 25310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:34.544437 25310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:34.544649 25310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:34.554999 25310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:45135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=35803
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:34.556694 25310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:34.558214 25310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:34.572736 25316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:34.572751 25315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:34.573568 25318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:34.573689 25310 server_base.cc:1034] running on GCE node
I20260807 19:34:34.574414 25310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:34.575508 25310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:34.576762 25310 hybrid_clock.cc:648] HybridClock initialized: now 1786131274576707 us; error 49 us; skew 500 ppm
I20260807 19:34:34.577132 25310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:34 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131274, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:34.580614 25310 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:34.582298 25310 webserver.cc:460] Webserver started at http://127.8.216.130:35803/ using document root <none> and password file <none>
I20260807 19:34:34.583081 25310 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:34.583248 25310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:34.589231 25310 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:34.592414 25325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:34.593917 25310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:34:34.594143 25310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "06833fae4c304fd091d5e90d3d214bd5"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "27eaae018a3115e11b8ae30b8ac0062f"
server_key_iv: "4f81ed7c5baeec504c20c20d184c5b24"
server_key_version: "encryptionkey@0"
I20260807 19:34:34.595219 25310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:34.617597 25310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:34.622370 25310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:34.622686 25310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:34.623816 25310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:34.625336 25310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:34.625479 25310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:34.625622 25310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:34.625713 25310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:34.662524 25310 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:45135
I20260807 19:34:34.662622 25448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:45135 every 8 connection(s)
I20260807 19:34:34.664242 25310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:34.668009 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25310
I20260807 19:34:34.668869 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:34715
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:36933
--webserver_port=43957
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:34:34.669955 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.667890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:56123)
0807 19:34:34.668728 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.668994 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":334,"threads_started":1}
W20260807 19:34:34.670783 25449 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:36933: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
I20260807 19:34:34.671772 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.671394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:53065)
0807 19:34:34.671571 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.671660 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:34:34.672837 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:34.672508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:59365)
0807 19:34:34.672645 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:34.672723 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:34:34.673156 25449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:34:35.051194 25452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:35.051537 25452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:35.051717 25452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:35.059700 25452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:35.059840 25452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:35.060038 25452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:35.071038 25452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:34715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=43957
--tserver_master_addrs=127.8.216.190:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:35.072551 25452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:35.073930 25452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:35.089412 25457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:35.089469 25458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:35.089494 25460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:35.090389 25452 server_base.cc:1034] running on GCE node
I20260807 19:34:35.091126 25452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:35.092283 25452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:35.093585 25452 hybrid_clock.cc:648] HybridClock initialized: now 1786131275093502 us; error 75 us; skew 500 ppm
I20260807 19:34:35.093993 25452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:35 dist-test-slave-8w9v krb5kdc[23868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131275, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:35.097899 25452 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:35.099772 25452 webserver.cc:460] Webserver started at http://127.8.216.131:43957/ using document root <none> and password file <none>
I20260807 19:34:35.100857 25452 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:35.101013 25452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:35.107615 25452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 19:34:35.110975 25468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:35.112767 25452 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:35.112953 25452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "1a9e42b690044ea9a6b57f7455f8c41e"
format_stamp: "Formatted at 2026-08-07 19:34:28 on dist-test-slave-8w9v"
server_key: "c10b17eb8eaf813bd8278e5c24538eab"
server_key_iv: "f7e1327e4719061bbd80cf6c3c42b417"
server_key_version: "encryptionkey@0"
I20260807 19:34:35.114073 25452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:35.147465 25452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:35.152594 25452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:35.152930 25452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:35.154043 25452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:35.155536 25452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:35.155627 25452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:35.155784 25452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:35.155882 25452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:35.163750 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.162596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:35039)
0807 19:34:35.163228 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.163382 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":258,"threads_started":1}
I20260807 19:34:35.193375 25452 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:34715
I20260807 19:34:35.193486 25590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:34715 every 8 connection(s)
I20260807 19:34:35.194964 25452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:35.200344 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.198128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:58225)
0807 19:34:35.199049 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.199310 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":402,"threads_started":1}
W20260807 19:34:35.201103 25591 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:36933 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:36933: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
I20260807 19:34:35.202024 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.201631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:40871)
0807 19:34:35.201823 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.201901 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:34:35.203023 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.202625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:46277)
0807 19:34:35.202799 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.202900 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 19:34:35.203518 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25452
W20260807 19:34:35.203786 25591 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:34:35.674880 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.673888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:38047)
0807 19:34:35.674480 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.674608 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":292,"threads_started":1}
I20260807 19:34:35.797972 25598 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:35.798393 25598 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:35.800411 25598 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715), 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
Aug 07 19:34:35 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131274, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
Aug 07 19:34:35 dist-test-slave-8w9v krb5kdc[23868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131274, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:34:35.825258 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.800969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:36787 (local address 127.8.216.131:34715)
0807 19:34:35.802031 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:35.802046 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:35.802094 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:34:35.803924 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:35.803946 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:35.804477 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:35.804873 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:35.807458 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.808777 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.810705 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.811386 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.811635 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:35.816821 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:35.816831 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:35.816843 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:35.816895 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:35.820727 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:35.821807 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:35.821814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:35.821820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:35.822026 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:35.822559 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:35.822565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:35.822568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:35.823214 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:35.823433 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:35.824364 (+ 931us) server_negotiation.cc:299] Negotiation successful
0807 19:34:35.824779 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":750,"threads_started":1}
I20260807 19:34:35.825258 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.801475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:40995 (local address 127.8.216.130:45135)
0807 19:34:35.802911 (+ 1436us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:35.802921 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:35.802965 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:34:35.804893 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:35.804905 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:35.805358 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:35.805725 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:35.807435 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.808792 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.810687 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.811386 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.811618 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:35.816692 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:35.816707 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:35.816730 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:35.816802 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:35.820679 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:35.821798 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:35.821809 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:35.821817 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:35.822019 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:35.822525 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:35.822530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:35.822534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:35.823190 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:35.823418 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:35.824365 (+ 947us) server_negotiation.cc:299] Negotiation successful
0807 19:34:35.824827 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1276,"thread_start_us":1103,"threads_started":1}
I20260807 19:34:35.825342 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.800831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:45135 (local address 127.8.216.129:40995)
0807 19:34:35.801672 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.801694 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:35.802140 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:35.805699 (+ 3559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:35.805703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:35.806356 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:35.807190 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:35.807213 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.809017 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.809022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:35.810503 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:35.810520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.811658 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.811666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:35.811990 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:35.812949 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:35.812981 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:35.816224 (+ 3243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:35.820893 (+ 4669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:35.820905 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:35.820935 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:35.821614 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:35.822159 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:35.822172 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:35.822175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:35.822392 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:35.823366 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:35.823375 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:35.823767 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 19:34:35.824085 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:34:35.824838 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":216,"spinlock_wait_cycles":130944,"thread_start_us":517,"threads_started":1}
I20260807 19:34:35.826198 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:35.800702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:34715 (local address 127.8.216.129:36787)
0807 19:34:35.801381 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:35.801426 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:35.802134 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:35.805221 (+ 3087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:35.805232 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:35.806359 (+ 1127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:35.807190 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:35.807214 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.809025 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.809029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:35.810503 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:35.810520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:35.811680 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:35.811687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:35.811995 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:35.812987 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:35.813006 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:35.816472 (+ 3466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:35.820911 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:35.820916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:35.820946 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:35.821629 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:35.822174 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:35.822178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:35.822181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:35.822386 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:35.823372 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:35.823376 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:35.823766 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 19:34:35.824085 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 19:34:35.825063 (+ 978us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":477,"spinlock_wait_cycles":16256,"thread_start_us":290,"threads_started":1}
W20260807 19:34:35.829761 25183 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
W20260807 19:34:35.830087 25184 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:35.830225 25184 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb035a92774e4370bc69c57c5b3cbf3b; no voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e
I20260807 19:34:35.830636 25598 raft_consensus.cc:2726] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:34:36.165587 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:36.165072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:42869)
0807 19:34:36.165323 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:36.165433 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 19:34:36.205683 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:36.204542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:45383)
0807 19:34:36.205189 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:36.205380 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":364,"threads_started":1}
I20260807 19:34:36.676817 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:36.675929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:55849)
0807 19:34:36.676558 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:36.676647 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":355,"threads_started":1}
I20260807 19:34:37.167491 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:37.166709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:60069)
0807 19:34:37.167206 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:37.167289 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":260,"threads_started":1}
I20260807 19:34:37.207675 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:37.206885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:48749)
0807 19:34:37.207408 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:37.207506 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
I20260807 19:34:37.678689 25607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:37.677887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:53263)
0807 19:34:37.678404 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:37.678503 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":261,"threads_started":1}
I20260807 19:34:37.840492 25608 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:37.840785 25608 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:37.841449 25608 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715), 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
W20260807 19:34:37.842662 25184 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
W20260807 19:34:37.842852 25183 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:37.843042 25183 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb035a92774e4370bc69c57c5b3cbf3b; no voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e
I20260807 19:34:37.843418 25608 raft_consensus.cc:2726] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:34:38.169523 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:38.168613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:58737)
0807 19:34:38.169203 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:38.169320 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":320,"threads_started":1}
I20260807 19:34:38.209611 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:38.208783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:38585)
0807 19:34:38.209323 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:38.209418 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":272,"threads_started":1}
I20260807 19:34:38.680840 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:38.679881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:47967)
0807 19:34:38.680528 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:38.680639 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":379,"threads_started":1}
I20260807 19:34:39.171574 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:39.170720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:48675)
0807 19:34:39.171272 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:39.171374 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":253,"threads_started":1}
I20260807 19:34:39.211678 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:39.210767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:52851)
0807 19:34:39.211379 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:39.211471 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":300,"threads_started":1}
I20260807 19:34:39.682833 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:39.682005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:58111)
0807 19:34:39.682544 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:39.682649 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":269,"threads_started":1}
I20260807 19:34:40.173463 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:40.172714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:36541)
0807 19:34:40.173187 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:40.173276 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260807 19:34:40.213915 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:40.213065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:57381)
0807 19:34:40.213641 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:40.213736 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":306,"threads_started":1}
I20260807 19:34:40.301234 25617 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:40.301478 25617 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:40.302183 25617 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715), 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
W20260807 19:34:40.303339 25183 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
W20260807 19:34:40.303606 25184 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:40.303778 25184 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb035a92774e4370bc69c57c5b3cbf3b; no voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e
I20260807 19:34:40.304015 25617 raft_consensus.cc:2726] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:34:40.684679 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:40.683788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:41643)
0807 19:34:40.684387 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:40.684481 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":278,"threads_started":1}
I20260807 19:34:41.175279 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:41.174457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:38005)
0807 19:34:41.174960 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:41.175053 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
I20260807 19:34:41.215726 25620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:41.214902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:56213)
0807 19:34:41.215454 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:41.215546 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":276,"threads_started":1}
I20260807 19:34:41.686745 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:41.685735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:40775)
0807 19:34:41.686402 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:41.686525 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":314,"threads_started":1}
I20260807 19:34:42.177280 25622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:42.176361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:51669)
0807 19:34:42.176887 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:42.176983 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260807 19:34:42.218098 25623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:42.217060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:38849)
0807 19:34:42.217772 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:42.217893 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":278,"threads_started":1}
I20260807 19:34:42.688836 25624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:42.687848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:40101)
0807 19:34:42.688495 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:42.688622 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":386,"threads_started":1}
I20260807 19:34:43.179433 25625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:43.178475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:57809)
0807 19:34:43.179041 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:43.179150 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":293,"threads_started":1}
I20260807 19:34:43.220029 25626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:43.219276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:46685)
0807 19:34:43.219748 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:43.219839 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":247,"threads_started":1}
I20260807 19:34:43.691154 25627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:43.690210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:42483)
0807 19:34:43.690840 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:43.690968 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":286,"threads_started":1}
I20260807 19:34:44.181515 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:44.180701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:33757)
0807 19:34:44.181237 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:44.181339 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":298,"threads_started":1}
I20260807 19:34:44.222075 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:44.221086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:38547)
0807 19:34:44.221678 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:44.221821 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":292,"threads_started":1}
I20260807 19:34:44.693017 25630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:44.692292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:36947)
0807 19:34:44.692744 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:44.692831 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":239,"threads_started":1}
I20260807 19:34:44.888793 25631 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:44.889036 25631 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:44.889850 25631 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715), 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
W20260807 19:34:44.890794 25184 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
W20260807 19:34:44.891006 25183 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:44.891122 25183 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb035a92774e4370bc69c57c5b3cbf3b; no voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e
I20260807 19:34:44.891379 25631 raft_consensus.cc:2726] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:34:45.183594 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:45.182697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:52245)
0807 19:34:45.183257 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:45.183379 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":278,"threads_started":1}
I20260807 19:34:45.224104 25633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:45.223222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:38945)
0807 19:34:45.223798 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:45.223905 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":345,"threads_started":1}
I20260807 19:34:45.695039 25634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:45.694157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:54235)
0807 19:34:45.694776 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:45.694859 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":343,"threads_started":1}
I20260807 19:34:46.185425 25635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:46.184649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:55973)
0807 19:34:46.185145 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:46.185234 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":271,"threads_started":1}
I20260807 19:34:46.226161 25636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:46.225267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:59143)
0807 19:34:46.225869 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:46.225991 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":337,"threads_started":1}
I20260807 19:34:46.697441 25637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:46.696349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:48071)
0807 19:34:46.697016 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:46.697143 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":338,"threads_started":1}
I20260807 19:34:47.187255 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:47.186492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:36361)
0807 19:34:47.186963 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:47.187053 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
I20260807 19:34:47.228458 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:47.227437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:40901)
0807 19:34:47.228078 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:47.228247 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":313,"threads_started":1}
I20260807 19:34:47.699564 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:47.698523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:37135)
0807 19:34:47.699080 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:47.699223 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":234,"threads_started":1}
I20260807 19:34:47.770418 25641 raft_consensus.cc:482] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:47.770658 25641 raft_consensus.cc:504] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a9e42b690044ea9a6b57f7455f8c41e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 34715 } } peers { permanent_uuid: "06833fae4c304fd091d5e90d3d214bd5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 45135 } } peers { permanent_uuid: "eb035a92774e4370bc69c57c5b3cbf3b" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 38695 } }
I20260807 19:34:47.771410 25641 leader_election.cc:288] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715), 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135)
W20260807 19:34:47.772658 25183 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06833fae4c304fd091d5e90d3d214bd5 (127.8.216.130:45135): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
W20260807 19:34:47.772929 25184 leader_election.cc:341] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a9e42b690044ea9a6b57f7455f8c41e (127.8.216.131:34715): Not found: Tablet not found: b777056c044f4afc81b93a9419c29e6a
I20260807 19:34:47.773082 25184 leader_election.cc:302] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb035a92774e4370bc69c57c5b3cbf3b; no voters: 06833fae4c304fd091d5e90d3d214bd5, 1a9e42b690044ea9a6b57f7455f8c41e
I20260807 19:34:47.773382 25641 raft_consensus.cc:2726] T b777056c044f4afc81b93a9419c29e6a P eb035a92774e4370bc69c57c5b3cbf3b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:34:48.189705 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:48.188425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:51087)
0807 19:34:48.189123 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:48.189334 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":347,"threads_started":1}
I20260807 19:34:48.230549 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:48.229676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:56295)
0807 19:34:48.230194 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:48.230343 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":267,"threads_started":1}
I20260807 19:34:48.701785 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:48.700755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:51655)
0807 19:34:48.701361 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:48.701503 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":323,"threads_started":1}
I20260807 19:34:49.192061 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:49.191227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:43471)
0807 19:34:49.191738 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:49.191835 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":277,"threads_started":1}
I20260807 19:34:49.232620 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:49.231714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:39913)
0807 19:34:49.232284 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:49.232397 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":283,"threads_started":1}
I20260807 19:34:49.703888 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:49.702981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.130:43457)
0807 19:34:49.703573 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:49.703703 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":302,"threads_started":1}
I20260807 19:34:50.193904 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:50.193103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.129:54349)
0807 19:34:50.193619 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:50.193708 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":246,"threads_started":1}
I20260807 19:34:50.213300 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25126
I20260807 19:34:50.230471 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:50.214614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.8.216.129:38695)
0807 19:34:50.215537 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:50.215546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:50.215574 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:34:50.215640 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:50.215647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:50.215859 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:50.216093 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:50.217853 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.219020 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.220518 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.221139 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.221295 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:50.224158 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:50.224177 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:50.224199 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:50.224248 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:50.227207 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.228021 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.228029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.228032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.228250 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.228667 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.228673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.228676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.229104 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:50.229427 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:50.229772 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 19:34:50.230087 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":577,"threads_started":1}
I20260807 19:34:50.231788 25300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:50.234853 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:50.233947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:36933 (local address 127.8.216.131:60807)
0807 19:34:50.234504 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:50.234630 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:36933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":306,"threads_started":1}
I20260807 19:34:50.326396 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25126
I20260807 19:34:50.349583 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25310
I20260807 19:34:50.365761 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:50.350488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.8.216.130:45135)
0807 19:34:50.350982 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:50.350988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:50.351005 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:34:50.351065 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:50.351069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:50.351131 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:50.351227 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:50.353001 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.353811 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.355293 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.355752 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.355862 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:50.359115 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:50.359119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:50.359123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:50.359166 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:50.362152 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.363043 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.363048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.363051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.363237 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.363754 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.363759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.363762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.364215 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:50.364494 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:50.364900 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:34:50.365141 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":288,"threads_started":1}
I20260807 19:34:50.366729 25444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:50.457244 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25310
I20260807 19:34:50.479372 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25452
I20260807 19:34:50.495553 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:50.480417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.8.216.131:34715)
0807 19:34:50.481104 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:50.481109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:50.481126 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:34:50.481185 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:50.481189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:50.481258 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:50.481392 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:50.483222 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.484084 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.485710 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:50.486245 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:50.486396 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:50.489514 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:50.489520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:50.489524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:50.489565 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:50.492660 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.493431 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.493438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.493442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.493603 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:50.494006 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:50.494012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:50.494015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:50.494302 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:50.494608 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:50.494897 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 19:34:50.495151 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":404,"threads_started":1}
I20260807 19:34:50.496419 25586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:50.585624 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25452
2026-08-07T19:34:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25713 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:34:50 dist-test-slave-8w9v krb5kdc[25659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:34:50 dist-test-slave-8w9v krb5kdc[25659](info): set up 2 sockets
Aug 07 19:34:50 dist-test-slave-8w9v krb5kdc[25659](info): commencing operation
krb5kdc: starting...
W20260807 19:34:51.771315 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:34:51 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:34:52.097756 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:52.469583 25675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:52.469955 25675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:52.470139 25675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:52.477880 25675 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:34:52.478060 25675 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:34:52.478160 25675 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:34:52.478251 25675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:34:52.478343 25675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:52.478466 25675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:52.478554 25675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:34:52.478641 25675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:34:52.489351 25675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:41295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:52.490793 25675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:52.492362 25675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:52.505527 25680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:52.505475 25681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:52.505537 25683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:52.506623 25675 server_base.cc:1034] running on GCE node
I20260807 19:34:52.507560 25675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:52.509253 25675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:52.510623 25675 hybrid_clock.cc:648] HybridClock initialized: now 1786131292510544 us; error 58 us; skew 500 ppm
I20260807 19:34:52.511032 25675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:52 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131292, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:52.518487 25675 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:34:52.520383 25675 webserver.cc:460] Webserver started at http://127.8.216.190:39123/ using document root <none> and password file <none>
I20260807 19:34:52.521168 25675 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:52.521299 25675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:52.521669 25675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:52.525080 25675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "05b5f97a1f29450cabc52ccc3f82a367"
format_stamp: "Formatted at 2026-08-07 19:34:52 on dist-test-slave-8w9v"
server_key: "1a98aeb3f887c3dc18504905860f3b09"
server_key_iv: "1adb476b3cfa09463ee773f0213b580c"
server_key_version: "encryptionkey@0"
I20260807 19:34:52.525974 25675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "05b5f97a1f29450cabc52ccc3f82a367"
format_stamp: "Formatted at 2026-08-07 19:34:52 on dist-test-slave-8w9v"
server_key: "1a98aeb3f887c3dc18504905860f3b09"
server_key_iv: "1adb476b3cfa09463ee773f0213b580c"
server_key_version: "encryptionkey@0"
I20260807 19:34:52.532424 25675 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:34:52.536651 25693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:52.538144 25675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:34:52.538374 25675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "05b5f97a1f29450cabc52ccc3f82a367"
format_stamp: "Formatted at 2026-08-07 19:34:52 on dist-test-slave-8w9v"
server_key: "1a98aeb3f887c3dc18504905860f3b09"
server_key_iv: "1adb476b3cfa09463ee773f0213b580c"
server_key_version: "encryptionkey@0"
I20260807 19:34:52.538619 25675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:52.566093 25675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:52.571412 25675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:52.571764 25675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:52.592609 25754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:41295 every 8 connection(s)
I20260807 19:34:52.592612 25675 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:41295
I20260807 19:34:52.594240 25675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:34:52.598476 25755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:52.601956 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25675
I20260807 19:34:52.602221 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:34:52.603176 9058 external_mini_cluster.cc:1428] Setting key 30b28499d2ade9f6327a632fac251123
I20260807 19:34:52.612517 25755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap starting.
I20260807 19:34:52.616169 25755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:52.617376 25755 log.cc:824] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:34:52 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:52.620792 25755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: No bootstrap required, opened a new log
I20260807 19:34:52.628598 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:52.605926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.8.216.190:41295)
0807 19:34:52.606816 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:52.606829 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:52.606878 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:34:52.608794 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:52.608822 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:52.609314 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:52.609790 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:52.611508 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:52.613036 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:52.614598 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:52.615517 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:52.615730 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:52.620660 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:52.620669 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:52.620685 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:52.620730 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:52.624168 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:52.624956 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:52.624964 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:52.624971 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:52.625152 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:52.625578 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:52.625583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:52.625590 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:52.626174 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:52.626511 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:52.627010 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 19:34:52.627465 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":412,"threads_started":1}
I20260807 19:34:52.629530 25755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:34:52.629976 25755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:52.630087 25755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Initialized, Role: FOLLOWER
I20260807 19:34:52.630729 25755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:34:52.630987 25755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:34:52.631132 25755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:34:52.631337 25755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:52.633857 25755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:34:52.634375 25755 leader_election.cc:302] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05b5f97a1f29450cabc52ccc3f82a367; no voters:
I20260807 19:34:52.634997 25755 leader_election.cc:288] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:34:52.635147 25760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:52.635887 25760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 LEADER]: Becoming Leader. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Running, Role: LEADER
I20260807 19:34:52.636656 25760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:34:52.637454 25755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:34:52.639719 25761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:34:52.640076 25761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:52.640291 25762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05b5f97a1f29450cabc52ccc3f82a367. Latest consensus state: current_term: 1 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:34:52.640662 25762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:34:52.644042 25766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:34:52.651212 25766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:34:52.661082 25766 catalog_manager.cc:1347] Generated new cluster ID: db755f90074b455c8d036d1be19c1f1d
I20260807 19:34:52.661234 25766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:34:52.684971 25766 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:34:52.686681 25766 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:34:52.695340 25766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Generated new TSK 0
I20260807 19:34:52.696297 25766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:34:52.773047 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:53.144716 25781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:53.145031 25781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:53.145188 25781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:53.153306 25781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:53.153451 25781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:53.153587 25781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:34:53.164093 25781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:53.165562 25781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:53.166951 25781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:53.181489 25787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:53.181504 25789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:53.182221 25786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:53.182201 25781 server_base.cc:1034] running on GCE node
I20260807 19:34:53.183197 25781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:53.184319 25781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:53.185556 25781 hybrid_clock.cc:648] HybridClock initialized: now 1786131293185483 us; error 65 us; skew 500 ppm
I20260807 19:34:53.185894 25781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:53 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131293, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:53.189517 25781 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:34:53.191247 25781 webserver.cc:460] Webserver started at http://127.8.216.129:43307/ using document root <none> and password file <none>
I20260807 19:34:53.192025 25781 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:53.192222 25781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:53.192584 25781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:53.196059 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "b0e7057c661d461d9facb2a8388be945"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "a45cea0ec88737466e0020940dcc8767"
server_key_iv: "27977149e7a402cc5b685be2b6a11b63"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.196943 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "b0e7057c661d461d9facb2a8388be945"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "a45cea0ec88737466e0020940dcc8767"
server_key_iv: "27977149e7a402cc5b685be2b6a11b63"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.202396 25781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:34:53.206121 25797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.207505 25781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:34:53.207690 25781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "b0e7057c661d461d9facb2a8388be945"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "a45cea0ec88737466e0020940dcc8767"
server_key_iv: "27977149e7a402cc5b685be2b6a11b63"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.207911 25781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:53.223829 25781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:53.228582 25781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:53.228899 25781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:53.229943 25781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:53.231462 25781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:53.231568 25781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.231716 25781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:53.231791 25781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.268610 25781 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37381
I20260807 19:34:53.268664 25919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37381 every 8 connection(s)
I20260807 19:34:53.270062 25781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:34:53.272608 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25781
I20260807 19:34:53.272850 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:34:53.273747 9058 external_mini_cluster.cc:1428] Setting key 8e76c024e2ad1d6c442a0abe27e6ad4d
Aug 07 19:34:53 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131293, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:53.293756 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:53.272947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49589 (local address 127.8.216.190:41295)
0807 19:34:53.273521 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:53.273529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:53.274914 (+ 1385us) server_negotiation.cc:407] Connection header received
0807 19:34:53.276700 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:53.276709 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:53.276814 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:53.276937 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:53.278564 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.279727 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.281724 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.282492 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.282654 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:53.286539 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:53.286549 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:53.286557 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:53.286608 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:53.289766 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:53.290920 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:53.290925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:53.290927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:53.291070 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:53.291664 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:53.291669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:53.291671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:53.291977 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:53.292109 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:53.293222 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0807 19:34:53.293480 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":261,"threads_started":1}
I20260807 19:34:53.295506 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:53.273582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:49589)
0807 19:34:53.274435 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:53.274484 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:53.276398 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:53.277156 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:53.277165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:53.277753 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:53.278365 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:53.278382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.279938 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.279942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:53.281561 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:53.281575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.282779 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.282785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:53.283025 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:53.283806 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:53.283832 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:53.286261 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:53.289959 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:53.289972 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:53.289996 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:53.290747 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:53.291190 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:53.291198 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:53.291204 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:53.291504 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:53.292182 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:53.292194 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:53.292725 (+ 531us) client_negotiation.cc:769] Sending connection context
0807 19:34:53.293065 (+ 340us) client_negotiation.cc:240] Negotiation successful
0807 19:34:53.293783 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":347,"threads_started":1}
I20260807 19:34:53.297508 25921 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:34:53.297895 25921 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:53.298745 25921 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:34:53.301304 25714 ts_manager.cc:194] Registered new tserver with Master: b0e7057c661d461d9facb2a8388be945 (127.8.216.129:37381)
I20260807 19:34:53.302863 25714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:49589
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:34:53.347434 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:53.714948 25927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:53.715457 25927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:53.715651 25927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:53.723767 25927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:53.723937 25927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:53.724108 25927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:34:53.734660 25927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:53.736093 25927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:53.737607 25927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:53.752498 25932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:53.752496 25933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:53.752889 25935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:53.754639 25927 server_base.cc:1034] running on GCE node
I20260807 19:34:53.755556 25927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:53.756724 25927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:53.758003 25927 hybrid_clock.cc:648] HybridClock initialized: now 1786131293757943 us; error 48 us; skew 500 ppm
I20260807 19:34:53.758450 25927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:53 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131293, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:53.762022 25927 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:34:53.763657 25927 webserver.cc:460] Webserver started at http://127.8.216.130:39119/ using document root <none> and password file <none>
I20260807 19:34:53.764511 25927 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:53.764667 25927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:53.765019 25927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:53.768394 25927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "63537300bb37c0b204d9bfefad25ec78"
server_key_iv: "5b9911388166fbe061a428b30891be7d"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.769294 25927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "63537300bb37c0b204d9bfefad25ec78"
server_key_iv: "5b9911388166fbe061a428b30891be7d"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.775252 25927 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:34:53.779285 25942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.780877 25927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:34:53.781100 25927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "63537300bb37c0b204d9bfefad25ec78"
server_key_iv: "5b9911388166fbe061a428b30891be7d"
server_key_version: "encryptionkey@0"
I20260807 19:34:53.781342 25927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:53.798197 25927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:53.803150 25927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:53.803504 25927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:53.804669 25927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:53.806138 25927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:53.806250 25927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.806403 25927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:53.806484 25927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:53.850772 25927 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:43105
I20260807 19:34:53.850813 26061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:43105 every 8 connection(s)
I20260807 19:34:53.852295 25927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:34:53.861065 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 25927
I20260807 19:34:53.861281 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:34:53.861877 9058 external_mini_cluster.cc:1428] Setting key 4979592a911dea982ef395c5870fc652
Aug 07 19:34:53 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131293, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:53.877514 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:53.855072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:41899 (local address 127.8.216.190:41295)
0807 19:34:53.855568 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:53.855573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:53.856656 (+ 1083us) server_negotiation.cc:407] Connection header received
0807 19:34:53.858614 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:53.858620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:53.858724 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:53.858845 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:53.860525 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.861677 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.863972 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.864544 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.864693 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:53.869108 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:53.869118 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:53.869126 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:53.869185 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:53.873400 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:53.874408 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:53.874417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:53.874424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:53.874607 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:53.875117 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:53.875125 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:53.875131 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:53.875594 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:53.875760 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:53.876674 (+ 914us) server_negotiation.cc:299] Negotiation successful
0807 19:34:53.877003 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":258,"threads_started":1}
I20260807 19:34:53.878504 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:53.855690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:41899)
0807 19:34:53.856482 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:53.856530 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:53.858323 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:53.859107 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:53.859116 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:53.859669 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:53.860344 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:53.860364 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.861889 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.861893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:53.863782 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:53.863800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:53.864835 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:53.864841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:53.865080 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:53.866005 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:53.866037 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:53.868761 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:53.873637 (+ 4876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:53.873647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:53.873663 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:53.874236 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:53.874736 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:53.874740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:53.874743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:53.874987 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:53.875737 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:53.875744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:53.876244 (+ 500us) client_negotiation.cc:769] Sending connection context
0807 19:34:53.876523 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 19:34:53.877002 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":298,"threads_started":1}
I20260807 19:34:53.880226 26063 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:34:53.880582 26063 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:53.881417 26063 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:34:53.883118 25714 ts_manager.cc:194] Registered new tserver with Master: 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105)
I20260807 19:34:53.883996 25714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:41899
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:34:53.935508 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:34:54.297286 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:34:54.297636 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:34:54.297816 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:34:54.305572 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:34:54.305753 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:34:54.305971 26071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:34:54.306205 25921 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:34:54.316830 26071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:34:54.318141 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:34:54.319582 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:34:54.333614 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:54.333680 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:34:54.333993 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:34:54.334867 26071 server_base.cc:1034] running on GCE node
I20260807 19:34:54.335620 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:34:54.336820 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:34:54.338074 26071 hybrid_clock.cc:648] HybridClock initialized: now 1786131294338016 us; error 45 us; skew 500 ppm
I20260807 19:34:54.338476 26071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:34:54 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:34:54.342164 26071 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:34:54.343884 26071 webserver.cc:460] Webserver started at http://127.8.216.131:45205/ using document root <none> and password file <none>
I20260807 19:34:54.344713 26071 fs_manager.cc:359] Metadata directory not provided
I20260807 19:34:54.344870 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:34:54.345180 26071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:34:54.348634 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "dd92576c342941a5bee9f749d928fe4d"
format_stamp: "Formatted at 2026-08-07 19:34:54 on dist-test-slave-8w9v"
server_key: "b5f6cd765dd0f168c0358766f85e3bb6"
server_key_iv: "15aeb2874dea196c394f7ba8dd3879bf"
server_key_version: "encryptionkey@0"
I20260807 19:34:54.349462 26071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "dd92576c342941a5bee9f749d928fe4d"
format_stamp: "Formatted at 2026-08-07 19:34:54 on dist-test-slave-8w9v"
server_key: "b5f6cd765dd0f168c0358766f85e3bb6"
server_key_iv: "15aeb2874dea196c394f7ba8dd3879bf"
server_key_version: "encryptionkey@0"
I20260807 19:34:54.355345 26071 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:34:54.359411 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:54.361459 26071 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:34:54.361640 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "dd92576c342941a5bee9f749d928fe4d"
format_stamp: "Formatted at 2026-08-07 19:34:54 on dist-test-slave-8w9v"
server_key: "b5f6cd765dd0f168c0358766f85e3bb6"
server_key_iv: "15aeb2874dea196c394f7ba8dd3879bf"
server_key_version: "encryptionkey@0"
I20260807 19:34:54.361837 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:34:54.383004 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:34:54.387722 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:34:54.387991 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:34:54.389106 26071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:34:54.390501 26071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:34:54.390578 26071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:54.390673 26071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:34:54.390717 26071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:34:54.427340 26071 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:37413
I20260807 19:34:54.427386 26205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:37413 every 8 connection(s)
I20260807 19:34:54.428740 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:34:54.437872 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26071
I20260807 19:34:54.438104 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:34:54.438609 9058 external_mini_cluster.cc:1428] Setting key 9fdce75c77fadb42ea1fad4cd274119c
Aug 07 19:34:54 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131294, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:34:54.451009 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.431396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:42363 (local address 127.8.216.190:41295)
0807 19:34:54.431930 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.431937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.432935 (+ 998us) server_negotiation.cc:407] Connection header received
0807 19:34:54.434776 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.434780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.434870 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.434969 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:54.436574 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.437547 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.439491 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.440089 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.440351 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.444666 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:54.444671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:54.444674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:54.444711 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:54.447699 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.448588 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.448592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.448594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.448741 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.449208 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.449246 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.449248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.449580 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:54.449725 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.450460 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.450673 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":241,"threads_started":1}
I20260807 19:34:54.452029 26209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.431952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:42363)
0807 19:34:54.432771 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.432816 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.434525 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.435174 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.435181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.435737 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:34:54.436393 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.436413 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.437713 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.437716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.439306 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.439323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.440437 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.440442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.440642 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.441504 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:34:54.441563 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:34:54.444454 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:34:54.447868 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:54.447876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:54.447889 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:54.448439 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:54.448854 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:34:54.448857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:34:54.448860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:34:54.449054 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:34:54.449701 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:34:54.449708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:34:54.450064 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.450319 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.450736 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":365,"threads_started":1}
I20260807 19:34:54.453769 26206 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:34:54.454123 26206 heartbeater.cc:464] Registering TS with master...
I20260807 19:34:54.455091 26206 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:34:54.456666 25714 ts_manager.cc:194] Registered new tserver with Master: dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
I20260807 19:34:54.457472 25714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:42363
I20260807 19:34:54.468215 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:34:54.487432 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.473486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.8.216.190:41295)
0807 19:34:54.473781 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.473789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.474154 (+ 365us) server_negotiation.cc:407] Connection header received
0807 19:34:54.474311 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.474316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.474393 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.474494 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:54.476075 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.477064 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.478466 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.478908 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.479015 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.481634 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:54.481639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:54.481642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:54.481679 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:54.484714 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.485443 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.485447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.485449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.485580 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.486016 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.486020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.486022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.486393 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:54.486577 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.486904 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.487091 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:34:54.491979 25714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:34:54.499066 25714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:34:54.536262 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.512809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.8.216.129:37381)
0807 19:34:54.525686 (+ 12877us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.525697 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.526984 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 19:34:54.527931 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.527940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.528219 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.529156 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:54.529398 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.530903 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.533631 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.534684 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.534830 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.535101 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.535614 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.535817 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12677,"thread_start_us":13855,"threads_started":1}
I20260807 19:34:54.536404 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.512662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.8.216.131:37413)
0807 19:34:54.526274 (+ 13612us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.526282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.526329 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:34:54.526428 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.526435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.526711 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.526920 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:54.528208 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.529871 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.533714 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.535201 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.535364 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.535594 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.535804 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.536000 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13409,"thread_start_us":13235,"threads_started":1}
I20260807 19:34:54.536834 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.512464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37413 (local address 127.0.0.1:60732)
0807 19:34:54.525331 (+ 12867us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.525410 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.525666 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.527144 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.527152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.527191 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:54.527647 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.527662 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.530555 (+ 2893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.530561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.533474 (+ 2913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.533484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.535424 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.535430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.535613 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.535647 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.535735 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.535946 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12463,"mutex_wait_us":2,"thread_start_us":13972,"threads_started":1}
I20260807 19:34:54.537369 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.512543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:37588)
0807 19:34:54.526764 (+ 14221us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.526788 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.526942 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.528479 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.528483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.528510 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:54.528943 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.528957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.531075 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.531089 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.533446 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.533462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.535075 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.535082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.535306 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.535375 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.536588 (+ 1213us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.536757 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14086,"mutex_wait_us":50,"thread_start_us":13396,"threads_started":1}
I20260807 19:34:54.541121 26138 tablet_service.cc:1467] Processing CreateTablet for tablet 42386f85cc2c42beac9cdea32bda562e (DEFAULT_TABLE table=test-table [id=cd80ffc74f78467a9add1151c5dd453e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:54.541203 25849 tablet_service.cc:1467] Processing CreateTablet for tablet 42386f85cc2c42beac9cdea32bda562e (DEFAULT_TABLE table=test-table [id=cd80ffc74f78467a9add1151c5dd453e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:54.542641 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.527131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43105 (local address 127.0.0.1:58392)
0807 19:34:54.531549 (+ 4418us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.531573 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.531706 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.532729 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.532739 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.532768 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:54.533234 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.533248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.536636 (+ 3388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.536641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.539238 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.539256 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.541661 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.541668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.541855 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.541917 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.542096 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.542247 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4264,"thread_start_us":4088,"threads_started":1}
I20260807 19:34:54.542773 25849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42386f85cc2c42beac9cdea32bda562e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:54.543017 26138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42386f85cc2c42beac9cdea32bda562e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:54.543768 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.527134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.8.216.130:43105)
0807 19:34:54.530451 (+ 3317us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.530460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.531866 (+ 1406us) server_negotiation.cc:407] Connection header received
0807 19:34:54.531978 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.531988 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.532274 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.532481 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:54.533446 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.536445 (+ 2999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.539851 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.541419 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.541579 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.541790 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.542918 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.543408 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3133,"thread_start_us":3004,"threads_started":1}
I20260807 19:34:54.547906 25991 tablet_service.cc:1467] Processing CreateTablet for tablet 42386f85cc2c42beac9cdea32bda562e (DEFAULT_TABLE table=test-table [id=cd80ffc74f78467a9add1151c5dd453e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:34:54.550156 25991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42386f85cc2c42beac9cdea32bda562e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:34:54.556002 26227 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Bootstrap starting.
I20260807 19:34:54.559410 26227 tablet_bootstrap.cc:654] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:54.560349 26228 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Bootstrap starting.
I20260807 19:34:54.560858 26227 log.cc:824] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:54.563910 26228 tablet_bootstrap.cc:654] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:54.564246 26227 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: No bootstrap required, opened a new log
I20260807 19:34:54.564559 26227 ts_tablet_manager.cc:1397] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260807 19:34:54.565141 26228 log.cc:824] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:54.567674 26232 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Bootstrap starting.
I20260807 19:34:54.571308 26228 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: No bootstrap required, opened a new log
I20260807 19:34:54.571640 26228 ts_tablet_manager.cc:1397] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 19:34:54.571656 26232 tablet_bootstrap.cc:654] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Neither blocks nor log segments found. Creating new log.
I20260807 19:34:54.573182 26232 log.cc:824] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Log is configured to *not* fsync() on all Append() calls
I20260807 19:34:54.574406 26227 raft_consensus.cc:348] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.574885 26227 raft_consensus.cc:374] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:54.575006 26227 raft_consensus.cc:729] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd92576c342941a5bee9f749d928fe4d, State: Initialized, Role: FOLLOWER
I20260807 19:34:54.575639 26227 consensus_queue.cc:260] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.576610 26232 tablet_bootstrap.cc:492] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: No bootstrap required, opened a new log
I20260807 19:34:54.576824 26206 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:34:54.576913 26232 ts_tablet_manager.cc:1397] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 19:34:54.578871 26227 ts_tablet_manager.cc:1428] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Time spent starting tablet: real 0.014s user 0.000s sys 0.012s
I20260807 19:34:54.582278 26228 raft_consensus.cc:348] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.582825 26228 raft_consensus.cc:374] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:54.582945 26228 raft_consensus.cc:729] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e7057c661d461d9facb2a8388be945, State: Initialized, Role: FOLLOWER
I20260807 19:34:54.583681 26228 consensus_queue.cc:260] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.585608 26228 ts_tablet_manager.cc:1428] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260807 19:34:54.589658 26232 raft_consensus.cc:348] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.590220 26232 raft_consensus.cc:374] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:34:54.590338 26232 raft_consensus.cc:729] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc713ff9c8d4979aeaef45aa6218dac, State: Initialized, Role: FOLLOWER
I20260807 19:34:54.590929 26232 consensus_queue.cc:260] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.592172 26063 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:34:54.592758 26232 ts_tablet_manager.cc:1428] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Time spent starting tablet: real 0.016s user 0.008s sys 0.005s
W20260807 19:34:54.604480 26064 tablet.cc:2369] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:34:54.680600 26207 tablet.cc:2369] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:54.700306 26235 raft_consensus.cc:482] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:34:54.700675 26235 raft_consensus.cc:504] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.702584 26235 leader_election.cc:288] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413), 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105)
I20260807 19:34:54.709522 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.703100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:54903 (local address 127.8.216.131:37413)
0807 19:34:54.703344 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.703351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.704159 (+ 808us) server_negotiation.cc:407] Connection header received
0807 19:34:54.704373 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.704378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.704457 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.704551 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:54.705121 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.705888 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.707731 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.708571 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.708756 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.708832 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.709116 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.709242 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 19:34:54.709707 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.703069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37413 (local address 127.8.216.129:54903)
0807 19:34:54.703997 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.704019 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.704227 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.704586 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.704591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.704632 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:54.704977 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.704986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.706015 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.706024 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.707569 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.707581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.708775 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.708781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.708950 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.708979 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.709087 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.709225 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":40,"spinlock_wait_cycles":5376,"thread_start_us":533,"threads_started":1}
I20260807 19:34:54.709748 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.703197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:45197 (local address 127.8.216.130:43105)
0807 19:34:54.703432 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.703455 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.704603 (+ 1148us) server_negotiation.cc:407] Connection header received
0807 19:34:54.704814 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.704824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.704884 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.704992 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:34:54.705529 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.706249 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.707994 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.708813 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.708999 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.709075 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.709349 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.709480 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:34:54.710246 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.703173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43105 (local address 127.8.216.129:45197)
0807 19:34:54.704476 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0807 19:34:54.704490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:34:54.704665 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:34:54.705006 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:34:54.705010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:34:54.705033 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:34:54.705354 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.705362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.706386 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.706393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.707823 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:34:54.707835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.709029 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.709036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:34:54.709189 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.709213 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:34:54.709310 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:34:54.709447 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1190,"mutex_wait_us":1,"spinlock_wait_cycles":5504,"thread_start_us":489,"threads_started":1}
I20260807 19:34:54.710938 26158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42386f85cc2c42beac9cdea32bda562e" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd92576c342941a5bee9f749d928fe4d" is_pre_election: true
I20260807 19:34:54.711180 26015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42386f85cc2c42beac9cdea32bda562e" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" is_pre_election: true
I20260807 19:34:54.711485 26158 raft_consensus.cc:2454] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 0.
I20260807 19:34:54.711819 26015 raft_consensus.cc:2454] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 0.
I20260807 19:34:54.712337 25800 leader_election.cc:302] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0e7057c661d461d9facb2a8388be945, dd92576c342941a5bee9f749d928fe4d; no voters:
I20260807 19:34:54.712813 26235 raft_consensus.cc:2781] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:34:54.712971 26235 raft_consensus.cc:482] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:34:54.713088 26235 raft_consensus.cc:3037] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:54.715397 26235 raft_consensus.cc:504] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.716079 26235 leader_election.cc:288] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 election: Requested vote from peers dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413), 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105)
I20260807 19:34:54.716688 26158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42386f85cc2c42beac9cdea32bda562e" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd92576c342941a5bee9f749d928fe4d"
I20260807 19:34:54.716696 26015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42386f85cc2c42beac9cdea32bda562e" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
I20260807 19:34:54.716914 26015 raft_consensus.cc:3037] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:54.716923 26158 raft_consensus.cc:3037] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:34:54.720347 26158 raft_consensus.cc:2454] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 1.
I20260807 19:34:54.720348 26015 raft_consensus.cc:2454] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 1.
I20260807 19:34:54.720888 25800 leader_election.cc:302] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6dc713ff9c8d4979aeaef45aa6218dac, b0e7057c661d461d9facb2a8388be945, dd92576c342941a5bee9f749d928fe4d; no voters:
I20260807 19:34:54.721290 26235 raft_consensus.cc:2781] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:34:54.721812 26235 raft_consensus.cc:686] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 1 LEADER]: Becoming Leader. State: Replica: b0e7057c661d461d9facb2a8388be945, State: Running, Role: LEADER
I20260807 19:34:54.722427 26235 consensus_queue.cc:237] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } }
I20260807 19:34:54.727686 25714 catalog_manager.cc:5495] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0e7057c661d461d9facb2a8388be945 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "b0e7057c661d461d9facb2a8388be945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } health_report { overall_health: UNKNOWN } } }
W20260807 19:34:54.772902 25922 tablet.cc:2369] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:34:54.783779 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.778221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.8.216.129:37381)
0807 19:34:54.778425 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.778430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.778457 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:34:54.778546 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.778549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.778640 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.778765 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:34:54.779216 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.780026 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.781566 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.781973 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.782078 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.782582 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:34:54.783111 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:34:54.783222 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.783304 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.783376 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:34:54.792579 26015 raft_consensus.cc:1261] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Refusing update from remote peer b0e7057c661d461d9facb2a8388be945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:54.792591 26158 raft_consensus.cc:1261] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Refusing update from remote peer b0e7057c661d461d9facb2a8388be945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:34:54.793582 26239 consensus_queue.cc:1035] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:54.793910 26235 consensus_queue.cc:1035] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:34:54.807169 26245 mvcc.cc:205] Tried to move back new op lower bound from 7315993783460200448 to 7315993783187316736. Current Snapshot: MvccSnapshot[applied={T|T < 7315993783460200448}]
I20260807 19:34:54.821214 25714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:54.821492 25714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:34:54.825497 25714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Sending DeleteTablet for 3 replicas of tablet 42386f85cc2c42beac9cdea32bda562e
I20260807 19:34:54.826776 26138 tablet_service.cc:1514] Processing DeleteTablet for tablet 42386f85cc2c42beac9cdea32bda562e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:60732
I20260807 19:34:54.826995 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25781
I20260807 19:34:54.827179 25991 tablet_service.cc:1514] Processing DeleteTablet for tablet 42386f85cc2c42beac9cdea32bda562e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:58392
I20260807 19:34:54.827773 25849 tablet_service.cc:1514] Processing DeleteTablet for tablet 42386f85cc2c42beac9cdea32bda562e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:34:54 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:37588
I20260807 19:34:54.839345 26254 tablet_replica.cc:330] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: stopping tablet replica
I20260807 19:34:54.839996 26254 raft_consensus.cc:2229] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:54.840723 26255 tablet_replica.cc:330] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: stopping tablet replica
I20260807 19:34:54.840751 26254 raft_consensus.cc:2258] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:54.841202 26255 raft_consensus.cc:2229] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:34:54.841802 26255 raft_consensus.cc:2258] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:54.841971 26256 tablet_replica.cc:330] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: stopping tablet replica
I20260807 19:34:54.842428 26256 raft_consensus.cc:2229] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:34:54.843273 26256 raft_consensus.cc:2258] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:34:54.844924 26255 ts_tablet_manager.cc:1905] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:54.846659 26256 ts_tablet_manager.cc:1905] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:54.850623 26254 ts_tablet_manager.cc:1905] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:34:54.853157 26255 ts_tablet_manager.cc:1918] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:54.853360 26255 log.cc:1196] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/42386f85cc2c42beac9cdea32bda562e
I20260807 19:34:54.853912 26255 ts_tablet_manager.cc:1939] T 42386f85cc2c42beac9cdea32bda562e P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting consensus metadata
I20260807 19:34:54.856073 25698 catalog_manager.cc:4841] TS 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): tablet 42386f85cc2c42beac9cdea32bda562e (table test-table [id=cd80ffc74f78467a9add1151c5dd453e]) successfully deleted
Aug 07 19:34:54 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:34:54.858464 26254 ts_tablet_manager.cc:1918] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:54.864068 26256 ts_tablet_manager.cc:1918] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:34:54.864315 26256 log.cc:1196] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/42386f85cc2c42beac9cdea32bda562e
I20260807 19:34:54.864897 26256 ts_tablet_manager.cc:1939] T 42386f85cc2c42beac9cdea32bda562e P b0e7057c661d461d9facb2a8388be945: Deleting consensus metadata
I20260807 19:34:54.863269 26254 log.cc:1196] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/42386f85cc2c42beac9cdea32bda562e
I20260807 19:34:54.867182 25698 catalog_manager.cc:4841] TS b0e7057c661d461d9facb2a8388be945 (127.8.216.129:37381): tablet 42386f85cc2c42beac9cdea32bda562e (table test-table [id=cd80ffc74f78467a9add1151c5dd453e]) successfully deleted
I20260807 19:34:54.867657 26254 ts_tablet_manager.cc:1939] T 42386f85cc2c42beac9cdea32bda562e P dd92576c342941a5bee9f749d928fe4d: Deleting consensus metadata
I20260807 19:34:54.869810 25696 catalog_manager.cc:4841] TS dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): tablet 42386f85cc2c42beac9cdea32bda562e (table test-table [id=cd80ffc74f78467a9add1151c5dd453e]) successfully deleted
I20260807 19:34:54.870103 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:54.829227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.8.216.129:37381)
0807 19:34:54.829486 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:54.829495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:54.841194 (+ 11699us) server_negotiation.cc:407] Connection header received
0807 19:34:54.841547 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:54.841557 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:54.841667 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:54.841796 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:54.844663 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.845807 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.848052 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:54.848839 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:54.848952 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:54.860805 (+ 11853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:54.860825 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:54.860846 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:54.860900 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:54.865675 (+ 4775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.866862 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.866871 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.866874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.867133 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:54.867751 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:54.867761 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:54.867764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:54.868304 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:54.868644 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:54.869336 (+ 692us) server_negotiation.cc:299] Negotiation successful
0807 19:34:54.869633 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:34:54.871472 25915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:54.977572 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25781
I20260807 19:34:55.003104 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25927
Aug 07 19:34:55 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:34:55.020088 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:55.003977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58400 (local address 127.8.216.130:43105)
0807 19:34:55.004229 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:55.004234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:55.004254 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:34:55.004388 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:55.004392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:55.004472 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:55.004565 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:55.005848 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:55.006655 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:55.008117 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:55.008633 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:55.008756 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:55.013662 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:55.013680 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:55.013698 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:55.013741 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:55.017029 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:55.017903 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:55.017912 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:55.017914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:55.018054 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:55.018434 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:55.018438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:55.018440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:55.018862 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:55.019094 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:55.019416 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 19:34:55.019707 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:34:55.020910 26057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:55.114598 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25927
I20260807 19:34:55.135928 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26071
Aug 07 19:34:55 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:34:55.152942 26224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:34:55.136865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.8.216.131:37413)
0807 19:34:55.137066 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:34:55.137070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:34:55.137095 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:34:55.137301 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:34:55.137306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:34:55.137384 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:34:55.137533 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:34:55.138962 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:55.139727 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:55.141072 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:34:55.141497 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:34:55.141614 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:34:55.146630 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:34:55.146660 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:34:55.146676 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:34:55.146723 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:34:55.149884 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:55.150735 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:55.150745 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:55.150747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:55.150910 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:34:55.151271 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:34:55.151274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:34:55.151276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:34:55.151679 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:34:55.151923 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 19:34:55.152325 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 19:34:55.152533 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:34:55.153721 26201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:55.247632 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26071
I20260807 19:34:55.269807 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 25675
I20260807 19:34:55.271359 25750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:34:55.388406 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 25675
I20260807 19:35:15.416872 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--webserver_interface=127.8.216.190
--webserver_port=39123
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:15.795655 26263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:15.796113 26263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:15.796372 26263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:15.804352 26263 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:35:15.804517 26263 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:35:15.804616 26263 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:35:15.804719 26263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:35:15.804826 26263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:15.804960 26263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:15.805052 26263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:35:15.805140 26263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:35:15.816179 26263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:41295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=39123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:15.817631 26263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:15.819129 26263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:15.847378 26272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:15.847390 26270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:15.847421 26269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:15.848342 26263 server_base.cc:1034] running on GCE node
I20260807 19:35:15.849316 26263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:15.851022 26263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:15.852300 26263 hybrid_clock.cc:648] HybridClock initialized: now 1786131315852217 us; error 43 us; skew 500 ppm
I20260807 19:35:15.852689 26263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:15 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131315, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:15.856617 26263 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:35:15.858340 26263 webserver.cc:460] Webserver started at http://127.8.216.190:39123/ using document root <none> and password file <none>
I20260807 19:35:15.859719 26263 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:15.859879 26263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:15.866945 26263 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:35:15.870661 26280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:15.872490 26263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:35:15.872680 26263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "05b5f97a1f29450cabc52ccc3f82a367"
format_stamp: "Formatted at 2026-08-07 19:34:52 on dist-test-slave-8w9v"
server_key: "1a98aeb3f887c3dc18504905860f3b09"
server_key_iv: "1adb476b3cfa09463ee773f0213b580c"
server_key_version: "encryptionkey@0"
I20260807 19:35:15.873847 26263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:15.909045 26263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:15.914397 26263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:15.914700 26263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:15.936291 26263 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:41295
I20260807 19:35:15.936265 26332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:41295 every 8 connection(s)
I20260807 19:35:15.938050 26263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:35:15.941699 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26263
I20260807 19:35:15.942600 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:37381
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=43307
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:35:15.946018 26333 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:35:15.949826 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap starting.
I20260807 19:35:15.978052 26333 log.cc:824] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:15.995199 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:35:15.995940 26333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap complete.
I20260807 19:35:16.006016 26333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:16.006984 26333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Initialized, Role: FOLLOWER
I20260807 19:35:16.007803 26333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:16.008082 26333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:35:16.008229 26333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:35:16.008404 26333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:35:16.012806 26333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:16.013429 26333 leader_election.cc:302] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05b5f97a1f29450cabc52ccc3f82a367; no voters:
I20260807 19:35:16.014168 26333 leader_election.cc:288] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:35:16.014325 26339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:35:16.015839 26339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 LEADER]: Becoming Leader. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Running, Role: LEADER
I20260807 19:35:16.016637 26339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:16.016922 26333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:35:16.019404 26342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05b5f97a1f29450cabc52ccc3f82a367. Latest consensus state: current_term: 2 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:35:16.019827 26342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:16.020484 26341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:35:16.020802 26341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:16.023092 26348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:35:16.032490 26348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:35:16.034137 26348 catalog_manager.cc:1259] Loaded cluster ID: db755f90074b455c8d036d1be19c1f1d
I20260807 19:35:16.034363 26348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:35:16.038671 26348 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:35:16.040568 26348 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Loaded TSK: 0
I20260807 19:35:16.073098 26348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Generated new TSK 1
I20260807 19:35:16.073551 26348 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:35:16.345299 26336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:16.345631 26336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:16.345813 26336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:16.353655 26336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:16.353796 26336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:16.354004 26336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:35:16.364750 26336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:37381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=43307
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:16.366353 26336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:16.367880 26336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:16.384825 26367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:16.384795 26368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:16.385363 26371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:16.385776 26336 server_base.cc:1034] running on GCE node
I20260807 19:35:16.386543 26336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:16.387714 26336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:16.388964 26336 hybrid_clock.cc:648] HybridClock initialized: now 1786131316388897 us; error 62 us; skew 500 ppm
I20260807 19:35:16.389325 26336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:16 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:16.393030 26336 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:35:16.394850 26336 webserver.cc:460] Webserver started at http://127.8.216.129:43307/ using document root <none> and password file <none>
I20260807 19:35:16.395678 26336 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:16.395867 26336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:16.402145 26336 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:35:16.405296 26378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.406785 26336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:35:16.406998 26336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "b0e7057c661d461d9facb2a8388be945"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "a45cea0ec88737466e0020940dcc8767"
server_key_iv: "27977149e7a402cc5b685be2b6a11b63"
server_key_version: "encryptionkey@0"
I20260807 19:35:16.408201 26336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:16.420753 26336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:16.425868 26336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:16.426143 26336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:16.427177 26336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:16.428720 26336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:16.428817 26336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.428901 26336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:16.428958 26336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.463373 26336 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37381
I20260807 19:35:16.463469 26497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37381 every 8 connection(s)
I20260807 19:35:16.465046 26336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:35:16.472357 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26336
I20260807 19:35:16.473047 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:43105
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=39119
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:35:16 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:16.494205 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:16.468532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:60125)
0807 19:35:16.469439 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:16.469495 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:16.471477 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:16.473639 (+ 2162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:16.473652 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:16.474393 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:16.475017 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:16.475039 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:16.477112 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:16.477126 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:16.478519 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:16.478537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:16.479919 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:16.479928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:16.480273 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:16.481122 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:16.481153 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:16.483734 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:16.488953 (+ 5219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:16.488963 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:16.488978 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:16.489545 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:16.490157 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:16.490160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:16.490162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:16.490370 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:16.491637 (+ 1267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:16.491645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:16.492021 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 19:35:16.492493 (+ 472us) client_negotiation.cc:240] Negotiation successful
0807 19:35:16.493013 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":404,"threads_started":1}
I20260807 19:35:16.494618 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:16.468351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:60125 (local address 127.8.216.190:41295)
0807 19:35:16.469040 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:16.469053 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:16.469669 (+ 616us) server_negotiation.cc:407] Connection header received
0807 19:35:16.472418 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:16.472458 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:16.472893 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:16.473434 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:16.475343 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:16.476879 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:16.478730 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:16.479591 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:16.479835 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:16.484190 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:16.484208 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:16.484238 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:16.484288 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:16.488720 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:16.489732 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:16.489745 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:16.489756 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:16.490021 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:16.490537 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:16.490545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:16.490552 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:16.491392 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:16.491634 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:16.492784 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0807 19:35:16.493307 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":316,"threads_started":1}
I20260807 19:35:16.499423 26498 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:35:16.499850 26498 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:16.500808 26498 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:35:16.503468 26297 ts_manager.cc:194] Registered new tserver with Master: b0e7057c661d461d9facb2a8388be945 (127.8.216.129:37381)
I20260807 19:35:16.505280 26297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:60125
W20260807 19:35:16.866500 26505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:16.866878 26505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:16.867051 26505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:16.875195 26505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:16.875389 26505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:16.875589 26505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:35:16.886952 26505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:43105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=39119
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:16.888536 26505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:16.890025 26505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:16.905179 26510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:16.905251 26511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:16.905149 26513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:16.906167 26505 server_base.cc:1034] running on GCE node
I20260807 19:35:16.906915 26505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:16.908041 26505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:16.909335 26505 hybrid_clock.cc:648] HybridClock initialized: now 1786131316909290 us; error 44 us; skew 500 ppm
I20260807 19:35:16.909734 26505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:16 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:16.913306 26505 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:35:16.915071 26505 webserver.cc:460] Webserver started at http://127.8.216.130:39119/ using document root <none> and password file <none>
I20260807 19:35:16.915844 26505 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:16.916038 26505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:16.922480 26505 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:35:16.926250 26521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.927872 26505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 19:35:16.928194 26505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "63537300bb37c0b204d9bfefad25ec78"
server_key_iv: "5b9911388166fbe061a428b30891be7d"
server_key_version: "encryptionkey@0"
I20260807 19:35:16.929391 26505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:16.947796 26505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:16.952723 26505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:16.953048 26505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:16.954106 26505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:16.955547 26505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:16.955673 26505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.955829 26505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:16.955931 26505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:16.992318 26505 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:43105
I20260807 19:35:16.992435 26653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:43105 every 8 connection(s)
I20260807 19:35:16.993945 26505 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:35:17.002072 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26505
I20260807 19:35:17.002782 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:37413
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=45205
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:35:17 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:17.019084 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:16.996741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:53089 (local address 127.8.216.190:41295)
0807 19:35:16.997273 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:16.997278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:16.998087 (+ 809us) server_negotiation.cc:407] Connection header received
0807 19:35:17.000048 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.000054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.000184 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.000311 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:17.002255 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.003157 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.005939 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.006390 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.006515 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.011978 (+ 5463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:17.011984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:17.011988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:17.012033 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:17.015176 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.016172 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.016179 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.016185 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.016331 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.016921 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.016933 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.016937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.017308 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:17.017503 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.018400 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.018669 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":274,"threads_started":1}
I20260807 19:35:17.020341 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:16.997176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:53089)
0807 19:35:16.997919 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:16.997960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:16.999746 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.000587 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.000597 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.001159 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:17.001951 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.001981 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.003375 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.003384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.005734 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.005756 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.006675 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.006685 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.007073 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.007938 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:17.007978 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:17.011438 (+ 3460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:17.015397 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:17.015409 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:17.015439 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:17.015977 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:17.016476 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:17.016482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:17.016486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:17.016777 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:17.017478 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:17.017488 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:17.017949 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.018319 (+ 370us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.018942 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":296,"threads_started":1}
I20260807 19:35:17.022105 26654 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:35:17.022473 26654 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:17.023494 26654 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:35:17.025503 26297 ts_manager.cc:194] Registered new tserver with Master: 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105)
I20260807 19:35:17.026832 26297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:53089
W20260807 19:35:17.383616 26658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:17.383939 26658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:17.384039 26658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:17.391727 26658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:17.391855 26658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:17.391973 26658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:35:17.402479 26658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:37413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=45205
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:17.403834 26658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:17.405507 26658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:17.419945 26665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:17.419926 26668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:17.420302 26666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:17.422027 26658 server_base.cc:1034] running on GCE node
I20260807 19:35:17.422734 26658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:17.423897 26658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:17.425161 26658 hybrid_clock.cc:648] HybridClock initialized: now 1786131317425098 us; error 38 us; skew 500 ppm
I20260807 19:35:17.425586 26658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:17 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131317, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:17.428915 26658 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:35:17.430594 26658 webserver.cc:460] Webserver started at http://127.8.216.131:45205/ using document root <none> and password file <none>
I20260807 19:35:17.431360 26658 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:17.431529 26658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:17.437443 26658 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:35:17.440402 26675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:17.441856 26658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:35:17.442059 26658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "dd92576c342941a5bee9f749d928fe4d"
format_stamp: "Formatted at 2026-08-07 19:34:54 on dist-test-slave-8w9v"
server_key: "b5f6cd765dd0f168c0358766f85e3bb6"
server_key_iv: "15aeb2874dea196c394f7ba8dd3879bf"
server_key_version: "encryptionkey@0"
I20260807 19:35:17.443126 26658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:17.464908 26658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:17.469576 26658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:17.469883 26658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:17.471004 26658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:17.472544 26658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:17.472671 26658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:17.472822 26658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:17.472914 26658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:17.509178 26498 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:35:17.511279 26658 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:37413
I20260807 19:35:17.511346 26796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:37413 every 8 connection(s)
I20260807 19:35:17.513034 26658 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:35:17.521247 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26658
Aug 07 19:35:17 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131317, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:17.539307 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.515986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:48443 (local address 127.8.216.190:41295)
0807 19:35:17.516224 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.516229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.517541 (+ 1312us) server_negotiation.cc:407] Connection header received
0807 19:35:17.519676 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.519682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.519778 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.519889 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:17.521756 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.522870 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.525575 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.526027 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.526176 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.530856 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:17.530862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:17.530866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:17.530907 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:17.535026 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.536317 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.536327 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.536333 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.536504 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.537100 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.537107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.537109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.537568 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:17.537748 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.538748 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.539002 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:35:17.540823 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.516525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:48443)
0807 19:35:17.517367 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.517411 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.519353 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.520172 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.520186 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.520762 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:17.521518 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.521549 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.523166 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.523176 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.525394 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.525409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.526313 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.526322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.526580 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.527466 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:17.527505 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:17.530564 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:17.535230 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:17.535242 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:17.535263 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:17.536105 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:17.536643 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:17.536651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:17.536658 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:17.536972 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:17.537724 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:17.537735 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:17.538279 (+ 544us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.538645 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.539321 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":341,"threads_started":1}
I20260807 19:35:17.542452 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.522516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.8.216.190:41295)
0807 19:35:17.524663 (+ 2147us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.524670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.525011 (+ 341us) server_negotiation.cc:407] Connection header received
0807 19:35:17.525260 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.525264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.525332 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.525448 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:17.527411 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.528723 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.530388 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.531077 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.531187 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.535228 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:17.535235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:17.535240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:17.535285 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:17.539162 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.540051 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.540057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.540060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.540241 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.540665 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.540669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.540672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.541070 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:17.541333 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.541751 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.542018 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1997,"thread_start_us":1830,"threads_started":1}
I20260807 19:35:17.542760 26799 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41295
I20260807 19:35:17.543128 26799 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:17.543932 26799 heartbeater.cc:511] Master 127.8.216.190:41295 requested a full tablet report, sending...
I20260807 19:35:17.545740 26297 ts_manager.cc:194] Registered new tserver with Master: dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
I20260807 19:35:17.546557 26297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:48443
I20260807 19:35:17.558353 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:35:17.566134 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.559542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57596 (local address 127.8.216.190:41295)
0807 19:35:17.559779 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.559785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.560171 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:35:17.560480 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.560485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.560642 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.560827 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:35:17.561329 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.562086 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.563749 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.564285 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.564432 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.565031 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:35:17.565393 (+ 362us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 19:35:17.565826 (+ 433us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57596: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 19:35:17.566272 26802 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57596: token expired
W20260807 19:35:17.566933 26211 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.216.190:41295)
I20260807 19:35:17.581648 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.567484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.8.216.190:41295)
0807 19:35:17.567673 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.567679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.567780 (+ 101us) server_negotiation.cc:407] Connection header received
0807 19:35:17.568006 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.568011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.568075 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.568216 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:17.569732 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.570564 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.572095 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.572600 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.572753 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.575706 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:17.575711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:17.575714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:17.575759 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:17.578830 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.579692 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.579697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.579699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.579843 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:17.580264 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:17.580268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:17.580270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:17.580652 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:17.580848 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.581175 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.581375 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:35:17.586046 26297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:35:17.594790 26297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:35:17.620365 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.608997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.8.216.129:37381)
0807 19:35:17.609639 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.609650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.610341 (+ 691us) server_negotiation.cc:407] Connection header received
0807 19:35:17.610490 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.610501 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.610824 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.611047 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:35:17.612405 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.614213 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.617450 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.618974 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.619152 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.619412 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.619818 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.619996 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":996,"threads_started":1}
I20260807 19:35:17.620419 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.608708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39198)
0807 19:35:17.609834 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.609894 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.610112 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.611415 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.611426 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.611472 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:35:17.612103 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.612158 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.614355 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.614362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.617223 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.617240 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.619230 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.619237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.619464 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.619520 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.619711 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.619929 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":2,"thread_start_us":905,"threads_started":1}
I20260807 19:35:17.621423 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.609452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37413 (local address 127.0.0.1:60390)
0807 19:35:17.610109 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.610127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.610240 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.614051 (+ 3811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.614056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.614084 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:35:17.614548 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.614565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.616348 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.616353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.618583 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.618598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.620620 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.620626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.620806 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.620840 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.620912 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.621062 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":82,"thread_start_us":876,"threads_started":1}
I20260807 19:35:17.622730 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.609726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.8.216.131:37413)
0807 19:35:17.613426 (+ 3700us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.613435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.613474 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:35:17.613576 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.613586 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.613832 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.614833 (+ 1001us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:35:17.615091 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.616204 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.618787 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.620346 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.621875 (+ 1529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.622116 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.622198 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.622370 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3478,"thread_start_us":3291,"threads_started":1}
I20260807 19:35:17.622759 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.610504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43105 (local address 127.0.0.1:54680)
0807 19:35:17.612055 (+ 1551us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.612090 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.612277 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.614547 (+ 2270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.614552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.614584 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:35:17.615062 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.615074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.617310 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.617316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.619588 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.619603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.621532 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.621538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.621731 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.621776 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.622162 (+ 386us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.622305 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"mutex_wait_us":1,"thread_start_us":1173,"threads_started":1}
I20260807 19:35:17.623623 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.610985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54680 (local address 127.8.216.130:43105)
0807 19:35:17.612869 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.612882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.612914 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:35:17.613046 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.613055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.613320 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.615273 (+ 1953us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:35:17.615525 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.617099 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.619756 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.621266 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.621442 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.621656 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.623107 (+ 1451us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.623299 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1723,"thread_start_us":1591,"threads_started":1}
I20260807 19:35:17.624992 26426 tablet_service.cc:1467] Processing CreateTablet for tablet 50cf4e1da619499ab835ffb215723264 (DEFAULT_TABLE table=test-table [id=92a598bbd344408fba1c937e3611f8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:35:17.626298 26426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50cf4e1da619499ab835ffb215723264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:35:17.626729 26727 tablet_service.cc:1467] Processing CreateTablet for tablet 50cf4e1da619499ab835ffb215723264 (DEFAULT_TABLE table=test-table [id=92a598bbd344408fba1c937e3611f8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:35:17.627110 26578 tablet_service.cc:1467] Processing CreateTablet for tablet 50cf4e1da619499ab835ffb215723264 (DEFAULT_TABLE table=test-table [id=92a598bbd344408fba1c937e3611f8e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:35:17.628291 26727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50cf4e1da619499ab835ffb215723264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:35:17.628361 26578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50cf4e1da619499ab835ffb215723264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:35:17.639798 26815 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Bootstrap starting.
I20260807 19:35:17.644438 26815 tablet_bootstrap.cc:654] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Neither blocks nor log segments found. Creating new log.
I20260807 19:35:17.645108 26816 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Bootstrap starting.
I20260807 19:35:17.646119 26815 log.cc:824] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:17.647181 26817 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Bootstrap starting.
I20260807 19:35:17.649212 26816 tablet_bootstrap.cc:654] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Neither blocks nor log segments found. Creating new log.
I20260807 19:35:17.650081 26815 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: No bootstrap required, opened a new log
I20260807 19:35:17.650388 26815 ts_tablet_manager.cc:1397] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260807 19:35:17.651407 26817 tablet_bootstrap.cc:654] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Neither blocks nor log segments found. Creating new log.
I20260807 19:35:17.650880 26816 log.cc:824] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:17.652778 26817 log.cc:824] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:17.654814 26816 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: No bootstrap required, opened a new log
I20260807 19:35:17.655138 26816 ts_tablet_manager.cc:1397] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260807 19:35:17.655956 26817 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: No bootstrap required, opened a new log
I20260807 19:35:17.656271 26817 ts_tablet_manager.cc:1397] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:35:17.659063 26815 raft_consensus.cc:348] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.659497 26815 raft_consensus.cc:374] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:35:17.659644 26815 raft_consensus.cc:729] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e7057c661d461d9facb2a8388be945, State: Initialized, Role: FOLLOWER
I20260807 19:35:17.660396 26815 consensus_queue.cc:260] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.662424 26815 ts_tablet_manager.cc:1428] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 19:35:17.667001 26817 raft_consensus.cc:348] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.667026 26816 raft_consensus.cc:348] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.667464 26817 raft_consensus.cc:374] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:35:17.667481 26816 raft_consensus.cc:374] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:35:17.667589 26816 raft_consensus.cc:729] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc713ff9c8d4979aeaef45aa6218dac, State: Initialized, Role: FOLLOWER
I20260807 19:35:17.667591 26817 raft_consensus.cc:729] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd92576c342941a5bee9f749d928fe4d, State: Initialized, Role: FOLLOWER
I20260807 19:35:17.668298 26816 consensus_queue.cc:260] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.668316 26817 consensus_queue.cc:260] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.669904 26799 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:35:17.670519 26817 ts_tablet_manager.cc:1428] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:35:17.671499 26654 heartbeater.cc:503] Master 127.8.216.190:41295 was elected leader, sending a full tablet report...
I20260807 19:35:17.672217 26816 ts_tablet_manager.cc:1428] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
W20260807 19:35:17.717911 26499 tablet.cc:2369] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:35:17.746397 26655 tablet.cc:2369] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:35:17.764909 26800 tablet.cc:2369] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:35:17.834280 26821 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:17.834662 26821 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.836544 26821 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
I20260807 19:35:17.843758 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.837025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:56533 (local address 127.8.216.130:43105)
0807 19:35:17.837249 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.837255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.838248 (+ 993us) server_negotiation.cc:407] Connection header received
0807 19:35:17.838466 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.838481 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.838547 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.838653 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:35:17.839287 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.840022 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.841962 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.842744 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.842955 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.843040 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.843325 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.843466 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:35:17.843804 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.837025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:47839 (local address 127.8.216.131:37413)
0807 19:35:17.837249 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.837256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.838363 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 19:35:17.838579 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.838586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.838659 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.838761 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:35:17.839429 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.840221 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.842027 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.842833 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.843037 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.843121 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.843412 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.843549 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:35:17.843940 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.837163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43105 (local address 127.8.216.129:56533)
0807 19:35:17.838121 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.838139 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.838284 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.838674 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.838678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.838704 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:35:17.839113 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.839125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.840210 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.840215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.841768 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.841784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.842971 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.842977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.843157 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.843182 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.843296 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.843453 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":24,"thread_start_us":627,"threads_started":1}
I20260807 19:35:17.843978 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.837163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37413 (local address 127.8.216.129:47839)
0807 19:35:17.838232 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:17.838252 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:17.838434 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:17.838786 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:17.838790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:17.838831 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:35:17.839252 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.839261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.840364 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.840370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.841864 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:17.841877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.843090 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.843097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:17.843247 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.843270 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:35:17.843373 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:35:17.843518 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":48,"spinlock_wait_cycles":22784,"thread_start_us":526,"threads_started":1}
I20260807 19:35:17.845279 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50cf4e1da619499ab835ffb215723264" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd92576c342941a5bee9f749d928fe4d" is_pre_election: true
I20260807 19:35:17.845279 26600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50cf4e1da619499ab835ffb215723264" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" is_pre_election: true
I20260807 19:35:17.845907 26600 raft_consensus.cc:2454] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 0.
I20260807 19:35:17.845907 26749 raft_consensus.cc:2454] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 0.
I20260807 19:35:17.846750 26382 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc713ff9c8d4979aeaef45aa6218dac, b0e7057c661d461d9facb2a8388be945; no voters:
I20260807 19:35:17.847232 26821 raft_consensus.cc:2781] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:35:17.847390 26821 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:35:17.847506 26821 raft_consensus.cc:3037] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:35:17.850612 26821 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.851341 26821 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 election: Requested vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
I20260807 19:35:17.851919 26749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50cf4e1da619499ab835ffb215723264" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd92576c342941a5bee9f749d928fe4d"
I20260807 19:35:17.851902 26600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50cf4e1da619499ab835ffb215723264" candidate_uuid: "b0e7057c661d461d9facb2a8388be945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
I20260807 19:35:17.852188 26600 raft_consensus.cc:3037] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:35:17.852188 26749 raft_consensus.cc:3037] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:35:17.855592 26749 raft_consensus.cc:2454] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 1.
I20260807 19:35:17.855592 26600 raft_consensus.cc:2454] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0e7057c661d461d9facb2a8388be945 in term 1.
I20260807 19:35:17.856097 26382 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dc713ff9c8d4979aeaef45aa6218dac, b0e7057c661d461d9facb2a8388be945; no voters:
I20260807 19:35:17.856530 26821 raft_consensus.cc:2781] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:35:17.857044 26821 raft_consensus.cc:686] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 LEADER]: Becoming Leader. State: Replica: b0e7057c661d461d9facb2a8388be945, State: Running, Role: LEADER
I20260807 19:35:17.857545 26821 consensus_queue.cc:237] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:17.862437 26297 catalog_manager.cc:5495] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0e7057c661d461d9facb2a8388be945 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "b0e7057c661d461d9facb2a8388be945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } health_report { overall_health: UNKNOWN } } }
I20260807 19:35:17.921969 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.915896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.8.216.129:37381)
0807 19:35:17.916099 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.916105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.916191 (+ 86us) server_negotiation.cc:407] Connection header received
0807 19:35:17.916507 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.916511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.916612 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.916747 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:35:17.917148 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.917948 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.919465 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.919915 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.920024 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:17.920720 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:35:17.921361 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:35:17.921481 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:17.921567 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 19:35:17.921644 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:35:17.930871 26600 raft_consensus.cc:1261] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Refusing update from remote peer b0e7057c661d461d9facb2a8388be945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:35:17.930887 26749 raft_consensus.cc:1261] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Refusing update from remote peer b0e7057c661d461d9facb2a8388be945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:35:17.932001 26826 consensus_queue.cc:1035] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:35:17.932469 26821 consensus_queue.cc:1035] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:35:17.946269 26828 mvcc.cc:205] Tried to move back new op lower bound from 7315993878234333184 to 7315993877948354560. Current Snapshot: MvccSnapshot[applied={T|T < 7315993878234333184}]
I20260807 19:35:17.967376 26297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:35:17.967952 26297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:35:17.973901 26297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Sending DeleteTablet for 3 replicas of tablet 50cf4e1da619499ab835ffb215723264
I20260807 19:35:17.975337 26727 tablet_service.cc:1514] Processing DeleteTablet for tablet 50cf4e1da619499ab835ffb215723264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:35:17 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:60390
I20260807 19:35:17.975443 26426 tablet_service.cc:1514] Processing DeleteTablet for tablet 50cf4e1da619499ab835ffb215723264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:35:17 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:39198
I20260807 19:35:17.975602 26578 tablet_service.cc:1514] Processing DeleteTablet for tablet 50cf4e1da619499ab835ffb215723264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:35:17 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:54680
I20260807 19:35:17.976166 26575 tablet_service.cc:1514] Processing DeleteTablet for tablet 50cf4e1da619499ab835ffb215723264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:35:17 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:54680
I20260807 19:35:17.982558 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26336
I20260807 19:35:18.032582 26838 tablet_replica.cc:330] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: stopping tablet replica
I20260807 19:35:18.035856 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:17.984332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.8.216.129:37381)
0807 19:35:17.985199 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:17.985209 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:17.985236 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:35:17.985327 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:17.985335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:17.985470 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:17.985593 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:17.988495 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:17.990272 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:17.998903 (+ 8631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:18.000052 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:18.000247 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:18.015104 (+ 14857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:18.015137 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:18.015161 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:18.015254 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:18.021730 (+ 6476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.023723 (+ 1993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.023740 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.023746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.024044 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.030017 (+ 5973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.030027 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.030033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.030878 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:18.031310 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:18.034557 (+ 3247us) server_negotiation.cc:299] Negotiation successful
0807 19:35:18.035010 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674}
I20260807 19:35:18.037053 26838 raft_consensus.cc:2229] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:35:18.038725 26838 raft_consensus.cc:2258] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:35:18.040215 26493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:18.040916 26840 tablet_replica.cc:330] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: stopping tablet replica
I20260807 19:35:18.041395 26840 raft_consensus.cc:2229] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:35:18.042009 26840 raft_consensus.cc:2258] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:35:18.046550 26840 ts_tablet_manager.cc:1905] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:35:18.048652 26838 ts_tablet_manager.cc:1905] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:35:18.053974 26840 ts_tablet_manager.cc:1918] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:35:18.054939 26840 log.cc:1196] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/50cf4e1da619499ab835ffb215723264
I20260807 19:35:18.057282 26840 ts_tablet_manager.cc:1939] T 50cf4e1da619499ab835ffb215723264 P 6dc713ff9c8d4979aeaef45aa6218dac: Deleting consensus metadata
I20260807 19:35:18.057945 26838 ts_tablet_manager.cc:1918] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:35:18.058274 26838 log.cc:1196] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/50cf4e1da619499ab835ffb215723264
I20260807 19:35:18.058938 26838 ts_tablet_manager.cc:1939] T 50cf4e1da619499ab835ffb215723264 P dd92576c342941a5bee9f749d928fe4d: Deleting consensus metadata
I20260807 19:35:18.060586 26284 catalog_manager.cc:4841] TS 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): tablet 50cf4e1da619499ab835ffb215723264 (table test-table [id=92a598bbd344408fba1c937e3611f8e2]) successfully deleted
I20260807 19:35:18.060756 26283 catalog_manager.cc:4841] TS dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): tablet 50cf4e1da619499ab835ffb215723264 (table test-table [id=92a598bbd344408fba1c937e3611f8e2]) successfully deleted
W20260807 19:35:18.061805 26284 catalog_manager.cc:4808] TS 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): delete failed for tablet 50cf4e1da619499ab835ffb215723264 because the tablet was not found. No further retry: Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:18.161892 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26336
I20260807 19:35:18.186532 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26505
W20260807 19:35:18.186501 26284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:37381 (error 108)
I20260807 19:35:18.189100 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.188102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39216)
0807 19:35:18.188780 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.188905 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":459,"threads_started":1}
W20260807 19:35:18.189399 26284 catalog_manager.cc:4584] TS b0e7057c661d461d9facb2a8388be945 (127.8.216.129:37381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50cf4e1da619499ab835ffb215723264: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
I20260807 19:35:18.202416 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.187458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.8.216.130:43105)
0807 19:35:18.187747 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:18.187752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:18.187770 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:35:18.187998 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:18.188006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:18.188104 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:18.188271 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:18.189788 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:18.190516 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:18.191859 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:18.192447 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:18.192549 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:18.195642 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:18.195661 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:18.195679 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:18.195718 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:18.199135 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.200019 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.200027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.200029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.200485 (+ 456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.200871 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.200874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.200877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.201294 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:18.201536 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:18.201804 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 19:35:18.202020 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260807 19:35:18.203226 26649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:18.243245 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.242611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39226)
0807 19:35:18.242915 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.243057 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 19:35:18.244572 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.244075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39228)
0807 19:35:18.244282 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.244377 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:35:18.298025 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26505
I20260807 19:35:18.305227 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.304751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39234)
0807 19:35:18.304973 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.305066 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:35:18.306360 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.305975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39236)
0807 19:35:18.306134 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.306214 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:35:18.320043 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26658
I20260807 19:35:18.334547 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.321021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.8.216.131:37413)
0807 19:35:18.321229 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:18.321234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:18.321254 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:35:18.321389 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:18.321394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:18.321465 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:18.321550 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:18.323150 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:18.323867 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:18.325392 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:18.325809 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:18.325929 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:18.328646 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:18.328664 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:18.328680 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:18.328726 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:18.331624 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.332491 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.332499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.332501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.332649 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:18.333012 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:18.333016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:18.333018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:18.333418 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:18.333649 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:18.333966 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:35:18.334150 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:35:18.335297 26792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:18.387492 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.386945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39238)
0807 19:35:18.387189 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.387287 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 19:35:18.388840 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.388365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39252)
0807 19:35:18.388597 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.388677 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 19:35:18.429343 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26658
I20260807 19:35:18.451768 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26263
I20260807 19:35:18.453223 26327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:18.580096 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:18.579389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:37381 (local address 127.0.0.1:39254)
0807 19:35:18.579817 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:18.579935 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260807 19:35:18.580225 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26263
I20260807 19:35:38.608552 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--webserver_interface=127.8.216.190
--webserver_port=39123
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:38.988019 26847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:38.988448 26847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:38.988703 26847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:38.996771 26847 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:35:38.996909 26847 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:35:38.997004 26847 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:35:38.997108 26847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:35:38.997192 26847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:38.997340 26847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:38.997401 26847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:35:38.997514 26847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:35:39.008621 26847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:41295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=39123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:39.010178 26847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:39.011749 26847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:39.026248 26855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:39.026415 26847 server_base.cc:1034] running on GCE node
W20260807 19:35:39.026384 26858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:39.026624 26852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:39.027783 26847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:39.029474 26847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:39.030781 26847 hybrid_clock.cc:648] HybridClock initialized: now 1786131339030709 us; error 50 us; skew 500 ppm
I20260807 19:35:39.031196 26847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:39 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131339, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:39.034883 26847 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:35:39.036630 26847 webserver.cc:460] Webserver started at http://127.8.216.190:39123/ using document root <none> and password file <none>
I20260807 19:35:39.037379 26847 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:39.037520 26847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:39.043678 26847 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 19:35:39.046756 26865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:39.048436 26847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:35:39.048619 26847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "05b5f97a1f29450cabc52ccc3f82a367"
format_stamp: "Formatted at 2026-08-07 19:34:52 on dist-test-slave-8w9v"
server_key: "1a98aeb3f887c3dc18504905860f3b09"
server_key_iv: "1adb476b3cfa09463ee773f0213b580c"
server_key_version: "encryptionkey@0"
I20260807 19:35:39.049631 26847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:39.068672 26847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:39.073678 26847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:39.074038 26847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:39.094067 26847 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:41295
I20260807 19:35:39.094091 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:41295 every 8 connection(s)
I20260807 19:35:39.095804 26847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:35:39.103451 26919 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:35:39.103989 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26847
I20260807 19:35:39.104904 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:37381
--local_ip_for_outbound_sockets=127.8.216.129
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=43307
--webserver_interface=127.8.216.129
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:35:39.107302 26919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap starting.
I20260807 19:35:39.133137 26919 log.cc:824] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:39.150534 26919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:35:39.151180 26919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Bootstrap complete.
I20260807 19:35:39.161427 26919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:39.162463 26919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Initialized, Role: FOLLOWER
I20260807 19:35:39.163162 26919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:39.163458 26919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:35:39.163585 26919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:35:39.163784 26919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:35:39.167927 26919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:39.168509 26919 leader_election.cc:302] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05b5f97a1f29450cabc52ccc3f82a367; no voters:
I20260807 19:35:39.169296 26919 leader_election.cc:288] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:35:39.169458 26923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:35:39.171032 26923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [term 3 LEADER]: Becoming Leader. State: Replica: 05b5f97a1f29450cabc52ccc3f82a367, State: Running, Role: LEADER
I20260807 19:35:39.171747 26923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } }
I20260807 19:35:39.172008 26919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:35:39.174786 26924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:35:39.175271 26924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:39.180019 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05b5f97a1f29450cabc52ccc3f82a367. Latest consensus state: current_term: 3 leader_uuid: "05b5f97a1f29450cabc52ccc3f82a367" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b5f97a1f29450cabc52ccc3f82a367" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41295 } } }
I20260807 19:35:39.181275 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367 [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:39.182935 26930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:35:39.192358 26930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:35:39.193943 26930 catalog_manager.cc:1259] Loaded cluster ID: db755f90074b455c8d036d1be19c1f1d
I20260807 19:35:39.194104 26930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:35:39.198551 26930 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:35:39.200361 26930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Loaded TSK: 0
I20260807 19:35:39.200609 26930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05b5f97a1f29450cabc52ccc3f82a367: Loaded TSK: 1
F20260807 19:35:39.201853 26930 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7efddd7e79ad google::LogMessage::Fail() at ??:0
@ 0x7efddd7ea88a google::LogMessage::SendToLog() at ??:0
@ 0x7efddd7e74de google::LogMessage::Flush() at ??:0
@ 0x7efddd7e8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7efde154b7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7efde153a73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7efdf32161c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7efdf3215783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7efdf32a12a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7efdf32a10c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7efdf32eb5e9 std::function<>::operator()() at ??:0
@ 0x7efdf3219cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7efdf3217c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7efdf329f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7efdf329f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efde0433345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7efde043691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7efde0436799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efde0411eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7efde36eb6db start_thread at ??:0
@ 0x7efddc68161f clone at ??:0
W20260807 19:35:39.562920 26921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:39.563225 26921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:39.563400 26921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:39.571420 26921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:39.571557 26921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:39.571738 26921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:35:39.583071 26921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:37381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=43307
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:39.585073 26921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:39.586525 26921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:39.601799 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:39.601663 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:39.601598 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:39.602607 26921 server_base.cc:1034] running on GCE node
I20260807 19:35:39.603327 26921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:39.604472 26921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:39.605700 26921 hybrid_clock.cc:648] HybridClock initialized: now 1786131339605626 us; error 62 us; skew 500 ppm
I20260807 19:35:39.606083 26921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:39 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131339, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:39.609798 26921 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:35:39.611486 26921 webserver.cc:460] Webserver started at http://127.8.216.129:43307/ using document root <none> and password file <none>
I20260807 19:35:39.612365 26921 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:39.612520 26921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:39.618816 26921 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:35:39.622063 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:39.623611 26921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:35:39.623831 26921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "b0e7057c661d461d9facb2a8388be945"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "a45cea0ec88737466e0020940dcc8767"
server_key_iv: "27977149e7a402cc5b685be2b6a11b63"
server_key_version: "encryptionkey@0"
I20260807 19:35:39.625015 26921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:39.644368 26921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:39.648960 26921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:39.649274 26921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:39.650394 26921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:39.652673 26988 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:35:39.659328 26921 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:35:39.659543 26921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 19:35:39.659722 26921 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:35:39.663514 26921 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:35:39.663650 26921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260807 19:35:39.664109 26988 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Bootstrap starting.
I20260807 19:35:39.692658 26988 log.cc:824] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:39.705819 26988 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:35:39.706736 26988 tablet_bootstrap.cc:492] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Bootstrap complete.
I20260807 19:35:39.708889 26988 ts_tablet_manager.cc:1397] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Time spent bootstrapping tablet: real 0.045s user 0.028s sys 0.015s
I20260807 19:35:39.719703 26921 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:37381
I20260807 19:35:39.719722 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:37381 every 8 connection(s)
I20260807 19:35:39.719888 26988 raft_consensus.cc:348] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:39.721160 26988 raft_consensus.cc:729] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0e7057c661d461d9facb2a8388be945, State: Initialized, Role: FOLLOWER
I20260807 19:35:39.721395 26921 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:35:39.721926 26988 consensus_queue.cc:260] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:39.724586 26988 ts_tablet_manager.cc:1428] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260807 19:35:39.725623 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 26921
I20260807 19:35:39.726406 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:43105
--local_ip_for_outbound_sockets=127.8.216.130
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=39119
--webserver_interface=127.8.216.130
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:35:39.727113 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:39.724721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:53161)
0807 19:35:39.725699 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:39.726003 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":447,"threads_started":1}
W20260807 19:35:39.727910 27121 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:41295 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:41295: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
I20260807 19:35:39.728978 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:39.728547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:57685)
0807 19:35:39.728716 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:39.728840 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:35:39.729943 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:39.729525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:49783)
0807 19:35:39.729684 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:39.729775 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 19:35:39.730144 27121 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:35:40.101385 27125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:40.101788 27125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:40.101977 27125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:40.110256 27125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:40.110456 27125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:40.110661 27125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:35:40.122699 27125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:43105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=39119
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:40.124367 27125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:40.125780 27125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:40.140539 27131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:40.140508 27134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:40.140475 27132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:40.141362 27125 server_base.cc:1034] running on GCE node
I20260807 19:35:40.142084 27125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:40.143112 27125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:40.144348 27125 hybrid_clock.cc:648] HybridClock initialized: now 1786131340144282 us; error 55 us; skew 500 ppm
I20260807 19:35:40.144698 27125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:40 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131340, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:40.148263 27125 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:35:40.149868 27125 webserver.cc:460] Webserver started at http://127.8.216.130:39119/ using document root <none> and password file <none>
I20260807 19:35:40.150591 27125 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:40.150695 27125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:40.156554 27125 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:35:40.159586 27144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.161172 27125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:35:40.161394 27125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "6dc713ff9c8d4979aeaef45aa6218dac"
format_stamp: "Formatted at 2026-08-07 19:34:53 on dist-test-slave-8w9v"
server_key: "63537300bb37c0b204d9bfefad25ec78"
server_key_iv: "5b9911388166fbe061a428b30891be7d"
server_key_version: "encryptionkey@0"
I20260807 19:35:40.162524 27125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:40.184695 27125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:40.189795 27125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:40.190178 27125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:40.191390 27125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:40.192929 27125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:40.193019 27125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.193233 27125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:40.193320 27125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.230044 27125 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:43105
I20260807 19:35:40.230154 27271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:43105 every 8 connection(s)
I20260807 19:35:40.231602 27125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:35:40.237071 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.234935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:59753)
0807 19:35:40.235753 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.236028 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":327,"threads_started":1}
W20260807 19:35:40.237995 27272 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:41295 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:41295: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
I20260807 19:35:40.239040 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.238639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:37119)
0807 19:35:40.238826 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.238910 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 19:35:40.240060 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.239705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:44631)
0807 19:35:40.239850 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.239941 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:35:40.240404 27272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:35:40.241348 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27125
I20260807 19:35:40.242007 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:37413
--local_ip_for_outbound_sockets=127.8.216.131
--tserver_master_addrs=127.8.216.190:41295
--webserver_port=45205
--webserver_interface=127.8.216.131
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:40.611162 27275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:40.611493 27275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:40.611598 27275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:40.619421 27275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:40.619560 27275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:40.619686 27275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:35:40.630364 27275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:37413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=45205
--tserver_master_addrs=127.8.216.190:41295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:40.631850 27275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:40.633340 27275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:40.647658 27284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:40.647711 27280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:40.647737 27282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:40.648590 27275 server_base.cc:1034] running on GCE node
I20260807 19:35:40.649384 27275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:40.650630 27275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:40.651890 27275 hybrid_clock.cc:648] HybridClock initialized: now 1786131340651814 us; error 67 us; skew 500 ppm
I20260807 19:35:40.652335 27275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:40 dist-test-slave-8w9v krb5kdc[25659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131340, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:40.656059 27275 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:35:40.657945 27275 webserver.cc:460] Webserver started at http://127.8.216.131:45205/ using document root <none> and password file <none>
I20260807 19:35:40.658983 27275 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:40.659152 27275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:40.665310 27275 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:35:40.668494 27292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.669993 27275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:35:40.670228 27275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "dd92576c342941a5bee9f749d928fe4d"
format_stamp: "Formatted at 2026-08-07 19:34:54 on dist-test-slave-8w9v"
server_key: "b5f6cd765dd0f168c0358766f85e3bb6"
server_key_iv: "15aeb2874dea196c394f7ba8dd3879bf"
server_key_version: "encryptionkey@0"
I20260807 19:35:40.671408 27275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:40.691659 27275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:40.696305 27275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:40.696645 27275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:40.697737 27275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:40.699227 27275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:40.699358 27275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.699505 27275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:40.699599 27275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:40.732086 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.730950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:48357)
0807 19:35:40.731595 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.731734 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":279,"threads_started":1}
I20260807 19:35:40.737133 27275 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:37413
I20260807 19:35:40.737229 27423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:37413 every 8 connection(s)
I20260807 19:35:40.738700 27275 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:35:40.743898 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.741928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:43415)
0807 19:35:40.742703 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.742973 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":293,"threads_started":1}
W20260807 19:35:40.744773 27424 heartbeater.cc:644] Failed to heartbeat to 127.8.216.190:41295 (0 consecutive failures): Network error: Failed to ping master at 127.8.216.190:41295: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
I20260807 19:35:40.745678 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.745316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:34387)
0807 19:35:40.745477 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.745560 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:35:40.746645 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:40.746327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:57447)
0807 19:35:40.746453 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:40.746530 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:35:40.746937 27424 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:35:40.747323 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27275
I20260807 19:35:41.242326 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.241167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:44871)
0807 19:35:41.241861 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:41.241991 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":360,"threads_started":1}
I20260807 19:35:41.493043 27429 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:41.493425 27429 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:41.495309 27429 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
Aug 07 19:35:41 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131339, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
Aug 07 19:35:41 dist-test-slave-8w9v krb5kdc[25659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131339, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:35:41.520251 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.496057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49431 (local address 127.8.216.131:37413)
0807 19:35:41.498021 (+ 1964us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:41.498035 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:41.498833 (+ 798us) server_negotiation.cc:407] Connection header received
0807 19:35:41.500452 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:41.500462 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:41.500881 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:41.501255 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:41.503514 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.504824 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.506659 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.507354 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.507578 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:41.512876 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:41.512883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:41.512896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:41.512939 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:41.515991 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517056 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:41.517072 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:41.517078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:41.517262 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517731 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:41.517738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:41.517746 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:41.518320 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:41.518560 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:41.519381 (+ 821us) server_negotiation.cc:299] Negotiation successful
0807 19:35:41.519782 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1786,"thread_start_us":1682,"threads_started":1}
I20260807 19:35:41.520282 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.495579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:43105 (local address 127.8.216.129:54753)
0807 19:35:41.496374 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:41.496435 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:41.497183 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:41.501409 (+ 4226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:41.501415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:41.502433 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:41.503275 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:41.503297 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.505056 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.505060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:41.506483 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:41.506498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.507592 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.507599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:41.507920 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:41.508944 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:41.508989 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:41.512311 (+ 3322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:41.516209 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:41.516219 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:41.516237 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:41.516880 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:41.517360 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:41.517369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:41.517590 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:41.518475 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:41.518483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:41.518843 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 19:35:41.519140 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 19:35:41.519747 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":214,"spinlock_wait_cycles":39552,"thread_start_us":1025,"threads_started":1}
I20260807 19:35:41.520800 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.495825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37413 (local address 127.8.216.129:49431)
0807 19:35:41.498688 (+ 2863us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:41.498707 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:41.498854 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:41.501397 (+ 2543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:41.501408 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:41.502432 (+ 1024us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:41.503276 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:41.503297 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.505036 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.505040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:41.506485 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:41.506499 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.507642 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.507651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:41.507924 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:41.508986 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:41.509006 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:41.512593 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:41.516209 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:41.516219 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:41.516242 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:41.516880 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:41.517388 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:41.517398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:41.517600 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:41.518476 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:41.518485 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:41.518843 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 19:35:41.519141 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 19:35:41.519930 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2726,"mutex_wait_us":406,"spinlock_wait_cycles":73984,"thread_start_us":2506,"threads_started":1}
I20260807 19:35:41.521508 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.496381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:54753 (local address 127.8.216.130:43105)
0807 19:35:41.498589 (+ 2208us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:41.498606 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:41.498654 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:35:41.500486 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:41.500495 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:41.500882 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:41.501230 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:41.503522 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.504824 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.506660 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:41.507342 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:41.507559 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:41.512678 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:41.512687 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:41.512703 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:41.512752 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:41.515981 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517055 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:41.517063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:41.517068 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:41.517236 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:41.517733 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:41.517737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:41.517739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:41.518320 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:41.518551 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:41.519390 (+ 839us) server_negotiation.cc:299] Negotiation successful
0807 19:35:41.521199 (+ 1809us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1993,"thread_start_us":1807,"threads_started":1}
W20260807 19:35:41.524482 26985 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
W20260807 19:35:41.524806 26983 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:41.524969 26983 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0e7057c661d461d9facb2a8388be945; no voters: 6dc713ff9c8d4979aeaef45aa6218dac, dd92576c342941a5bee9f749d928fe4d
I20260807 19:35:41.525404 27429 raft_consensus.cc:2726] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:35:41.733704 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.733229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:42441)
0807 19:35:41.733445 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:41.733549 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:35:41.748858 27434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:41.747649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:33091)
0807 19:35:41.748286 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:41.748500 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":331,"threads_started":1}
I20260807 19:35:42.244324 27436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:42.243471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:33747)
0807 19:35:42.244025 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:42.244141 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":284,"threads_started":1}
I20260807 19:35:42.735707 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:42.734713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:57863)
0807 19:35:42.735402 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:42.735503 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":371,"threads_started":1}
I20260807 19:35:42.750700 27438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:42.749849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:46185)
0807 19:35:42.750430 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:42.750526 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":330,"threads_started":1}
I20260807 19:35:43.246081 27439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:43.245297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:52497)
0807 19:35:43.245789 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:43.245888 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":252,"threads_started":1}
I20260807 19:35:43.585124 27440 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:43.585381 27440 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:43.586057 27440 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
W20260807 19:35:43.587270 26985 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
W20260807 19:35:43.587484 26983 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:43.587607 26983 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0e7057c661d461d9facb2a8388be945; no voters: 6dc713ff9c8d4979aeaef45aa6218dac, dd92576c342941a5bee9f749d928fe4d
I20260807 19:35:43.587895 27440 raft_consensus.cc:2726] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:35:43.737843 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:43.736865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:57789)
0807 19:35:43.737501 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:43.737628 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":322,"threads_started":1}
I20260807 19:35:43.752934 27442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:43.751811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:46345)
0807 19:35:43.752557 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:43.752680 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":416,"threads_started":1}
I20260807 19:35:44.248195 27443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:44.247276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:38843)
0807 19:35:44.247840 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:44.247960 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":283,"threads_started":1}
I20260807 19:35:44.739950 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:44.739121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:56929)
0807 19:35:44.739673 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:44.739763 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":275,"threads_started":1}
I20260807 19:35:44.755330 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:44.754437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:41695)
0807 19:35:44.755040 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:44.755139 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":329,"threads_started":1}
I20260807 19:35:45.250631 27446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:45.249527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:54805)
0807 19:35:45.250305 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:45.250421 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":464,"threads_started":1}
I20260807 19:35:45.510066 27447 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:45.510352 27447 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:45.511178 27447 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
W20260807 19:35:45.512357 26983 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
W20260807 19:35:45.512614 26985 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:45.512710 26985 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0e7057c661d461d9facb2a8388be945; no voters: 6dc713ff9c8d4979aeaef45aa6218dac, dd92576c342941a5bee9f749d928fe4d
I20260807 19:35:45.512969 27447 raft_consensus.cc:2726] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:35:45.741997 27448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:45.741057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:56741)
0807 19:35:45.741689 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:45.741794 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":372,"threads_started":1}
I20260807 19:35:45.757284 27451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:45.756529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:51231)
0807 19:35:45.757008 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:45.757109 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260807 19:35:46.252681 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:46.251754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:43177)
0807 19:35:46.252377 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:46.252481 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":295,"threads_started":1}
I20260807 19:35:46.743789 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:46.743002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:37863)
0807 19:35:46.743482 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:46.743585 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
I20260807 19:35:46.759378 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:46.758388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:42859)
0807 19:35:46.759085 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:46.759183 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":422,"threads_started":1}
I20260807 19:35:47.254804 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:47.253837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:42845)
0807 19:35:47.254491 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:47.254597 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":302,"threads_started":1}
I20260807 19:35:47.746058 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:47.744964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:54521)
0807 19:35:47.745670 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:47.745792 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":348,"threads_started":1}
I20260807 19:35:47.761410 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:47.760499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:47063)
0807 19:35:47.761094 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:47.761194 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":265,"threads_started":1}
I20260807 19:35:48.256978 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:48.256170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:36461)
0807 19:35:48.256667 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:48.256781 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":249,"threads_started":1}
I20260807 19:35:48.748044 27460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:48.747181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:54205)
0807 19:35:48.747752 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:48.747848 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":310,"threads_started":1}
I20260807 19:35:48.763397 27461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:48.762529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:44951)
0807 19:35:48.763102 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:48.763197 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":321,"threads_started":1}
I20260807 19:35:49.259124 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:49.258234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:32889)
0807 19:35:49.258841 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:49.258950 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":324,"threads_started":1}
I20260807 19:35:49.750496 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:49.749413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:46813)
0807 19:35:49.750186 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:49.750293 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":447,"threads_started":1}
I20260807 19:35:49.765487 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:49.764576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:51905)
0807 19:35:49.765211 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:49.765315 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":311,"threads_started":1}
I20260807 19:35:49.889897 27465 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:49.890157 27465 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:49.890956 27465 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
W20260807 19:35:49.891957 26985 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
W20260807 19:35:49.892159 26983 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:49.892282 26983 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0e7057c661d461d9facb2a8388be945; no voters: 6dc713ff9c8d4979aeaef45aa6218dac, dd92576c342941a5bee9f749d928fe4d
I20260807 19:35:49.892514 27465 raft_consensus.cc:2726] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:35:50.261368 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:50.260503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:51443)
0807 19:35:50.261032 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:50.261127 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":274,"threads_started":1}
I20260807 19:35:50.752430 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:50.751607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:51673)
0807 19:35:50.752073 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:50.752230 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260807 19:35:50.767297 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:50.766468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:46443)
0807 19:35:50.766975 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:50.767065 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":295,"threads_started":1}
I20260807 19:35:51.263401 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:51.262488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:52207)
0807 19:35:51.263110 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:51.263210 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":328,"threads_started":1}
I20260807 19:35:51.754222 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:51.753433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:53199)
0807 19:35:51.753903 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:51.753992 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":248,"threads_started":1}
I20260807 19:35:51.769057 27471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:51.768287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:58871)
0807 19:35:51.768786 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:51.768879 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
I20260807 19:35:52.265323 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:52.264567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:56009)
0807 19:35:52.265055 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:52.265142 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260807 19:35:52.756649 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:52.755552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:39559)
0807 19:35:52.756230 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:52.756360 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":327,"threads_started":1}
I20260807 19:35:52.771214 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:52.770149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:40261)
0807 19:35:52.770840 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:52.770981 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":414,"threads_started":1}
I20260807 19:35:53.267488 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:53.266500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:45853)
0807 19:35:53.267129 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:53.267239 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":321,"threads_started":1}
I20260807 19:35:53.758445 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:53.757619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:58319)
0807 19:35:53.758145 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:53.758243 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
I20260807 19:35:53.773434 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:53.772493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:53943)
0807 19:35:53.773102 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:53.773224 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":294,"threads_started":1}
I20260807 19:35:54.269364 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:54.268526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:54205)
0807 19:35:54.269072 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:54.269160 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
I20260807 19:35:54.760733 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:54.759807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:54235)
0807 19:35:54.760403 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:54.760511 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":334,"threads_started":1}
I20260807 19:35:54.775394 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:54.774499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:60973)
0807 19:35:54.774998 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:54.775180 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":272,"threads_started":1}
I20260807 19:35:55.271423 27482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:55.270701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.130:57147)
0807 19:35:55.271149 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:55.271260 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":216,"threads_started":1}
I20260807 19:35:55.436738 27483 raft_consensus.cc:482] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:35:55.436964 27483 raft_consensus.cc:504] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0e7057c661d461d9facb2a8388be945" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 37381 } } peers { permanent_uuid: "6dc713ff9c8d4979aeaef45aa6218dac" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 43105 } } peers { permanent_uuid: "dd92576c342941a5bee9f749d928fe4d" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37413 } }
I20260807 19:35:55.437788 27483 leader_election.cc:288] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105), dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413)
W20260807 19:35:55.438778 26983 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dd92576c342941a5bee9f749d928fe4d (127.8.216.131:37413): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
W20260807 19:35:55.438988 26985 leader_election.cc:341] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6dc713ff9c8d4979aeaef45aa6218dac (127.8.216.130:43105): Not found: Tablet not found: 50cf4e1da619499ab835ffb215723264
I20260807 19:35:55.439119 26985 leader_election.cc:302] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0e7057c661d461d9facb2a8388be945; no voters: 6dc713ff9c8d4979aeaef45aa6218dac, dd92576c342941a5bee9f749d928fe4d
I20260807 19:35:55.439426 27483 raft_consensus.cc:2726] T 50cf4e1da619499ab835ffb215723264 P b0e7057c661d461d9facb2a8388be945 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:35:55.754580 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 26921
I20260807 19:35:55.762857 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:55.761690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.129:53041)
0807 19:35:55.762375 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:55.762577 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":369,"threads_started":1}
I20260807 19:35:55.772652 27484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:55.757062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36098 (local address 127.8.216.129:37381)
0807 19:35:55.757611 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:55.757619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:55.757647 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:35:55.757711 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:55.757718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:55.758005 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:55.758185 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:55.759650 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:55.760789 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:55.762086 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:55.762765 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:55.762938 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:55.765977 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:55.765990 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:55.766006 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:55.766047 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:55.769465 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:55.770296 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:55.770304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:55.770307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:55.770456 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:55.770882 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:55.770888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:55.770891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:55.771271 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:55.771637 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:55.771979 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:35:55.772305 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":770,"threads_started":1}
I20260807 19:35:55.773985 27116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:55.778007 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:55.776932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41295 (local address 127.8.216.131:47439)
0807 19:35:55.777578 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:55.777731 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.190:41295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":288,"threads_started":1}
I20260807 19:35:55.869236 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 26921
I20260807 19:35:55.891333 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27125
I20260807 19:35:55.905319 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:55.892250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.8.216.130:43105)
0807 19:35:55.892754 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:55.892760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:55.892781 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:35:55.892844 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:55.892848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:55.892911 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:55.893009 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:55.894437 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:55.895320 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:55.896795 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:55.897221 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:55.897352 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:55.899929 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:55.899934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:55.899938 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:55.899973 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:55.902835 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:55.903481 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:55.903485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:55.903488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:55.903629 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:55.903969 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:55.903973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:55.903974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:55.904352 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:55.904558 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:55.904803 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 19:35:55.904979 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
I20260807 19:35:55.906111 27266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:55.993273 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27125
I20260807 19:35:56.014816 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27275
I20260807 19:35:56.029436 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:56.015670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.8.216.131:37413)
0807 19:35:56.016342 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:56.016347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:56.016364 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:35:56.016425 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:56.016429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:56.016499 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:56.016621 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:56.018158 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:56.018957 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:56.020352 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:56.020805 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:56.020976 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:56.023687 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:56.023692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:56.023696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:56.023733 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:56.026423 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:56.027189 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:56.027196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:56.027200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:56.027362 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:56.027780 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:56.027786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:56.027789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:56.028083 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:56.028462 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:56.028807 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 19:35:56.029008 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":311,"threads_started":1}
I20260807 19:35:56.030185 27416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:35:56.116890 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27275
2026-08-07T19:35:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65530 ms)
[----------] 30 tests from SecurityITest (316841 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:35:56 dist-test-slave-8w9v krb5kdc[27494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:35:56 dist-test-slave-8w9v krb5kdc[27494](info): set up 2 sockets
Aug 07 19:35:56 dist-test-slave-8w9v krb5kdc[27494](info): commencing operation
krb5kdc: starting...
W20260807 19:35:57.342607 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:35:57 dist-test-slave-8w9v krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:35:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:35:57.680481 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:37151
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:37151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:58.039832 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:58.040227 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:58.040357 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:58.047770 27508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:35:58.047885 27508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:58.047945 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:58.048002 27508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:35:58.048049 27508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:35:58.058313 27508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:37151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:37151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:58.059630 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:58.060959 27508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:58.073136 27515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:58.073622 27518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:58.073680 27514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:58.074191 27508 server_base.cc:1034] running on GCE node
I20260807 19:35:58.075035 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:58.076671 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:58.077924 27508 hybrid_clock.cc:648] HybridClock initialized: now 1786131358077874 us; error 35 us; skew 500 ppm
I20260807 19:35:58.078339 27508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:58 dist-test-slave-8w9v krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:58.086241 27508 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:35:58.088070 27508 webserver.cc:460] Webserver started at http://127.8.216.190:41723/ using document root <none> and password file <none>
I20260807 19:35:58.088935 27508 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:58.089097 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:58.089443 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:35:58.092773 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "e292ee3937e741ce82fe710dc81d575b"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "6b0a08d4226b5aad368ccc11562eb607"
server_key_iv: "45dd52add133d695ec4089c9b0a2a2ff"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.093583 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "e292ee3937e741ce82fe710dc81d575b"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "6b0a08d4226b5aad368ccc11562eb607"
server_key_iv: "45dd52add133d695ec4089c9b0a2a2ff"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.099123 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:35:58.102667 27525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:58.104199 27508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:35:58.104419 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "e292ee3937e741ce82fe710dc81d575b"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "6b0a08d4226b5aad368ccc11562eb607"
server_key_iv: "45dd52add133d695ec4089c9b0a2a2ff"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.104648 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:58.121019 27508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:58.125344 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:58.125669 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:58.145663 27508 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:37151
I20260807 19:35:58.145682 27581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:37151 every 8 connection(s)
I20260807 19:35:58.147305 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:35:58.151698 27582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:35:58.153213 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27508
I20260807 19:35:58.153472 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:35:58.154333 9058 external_mini_cluster.cc:1428] Setting key 412022fe084170871ca6e63b7c049c2d
I20260807 19:35:58.171069 27582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: Bootstrap starting.
Aug 07 19:35:58 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:58.174927 27582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: Neither blocks nor log segments found. Creating new log.
I20260807 19:35:58.176321 27582 log.cc:824] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:35:58.179468 27582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: No bootstrap required, opened a new log
I20260807 19:35:58.181631 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:58.156691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.8.216.190:37151)
0807 19:35:58.157729 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:58.157741 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:58.157840 (+ 99us) server_negotiation.cc:407] Connection header received
0807 19:35:58.159995 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:58.160025 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:58.160659 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:58.161071 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:58.163263 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.164705 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.166852 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.167692 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.167952 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:58.173361 (+ 5409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:58.173370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:58.173387 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:58.173442 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:58.177127 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:58.177946 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:58.177954 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:58.177961 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:58.178135 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:58.178596 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:58.178607 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:58.178610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:58.179189 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:58.179520 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:58.180031 (+ 511us) server_negotiation.cc:299] Negotiation successful
0807 19:35:58.180537 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":567,"threads_started":1}
I20260807 19:35:58.187973 27582 raft_consensus.cc:348] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } }
I20260807 19:35:58.188472 27582 raft_consensus.cc:374] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:35:58.188588 27582 raft_consensus.cc:729] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e292ee3937e741ce82fe710dc81d575b, State: Initialized, Role: FOLLOWER
I20260807 19:35:58.189229 27582 consensus_queue.cc:260] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } }
I20260807 19:35:58.189471 27582 raft_consensus.cc:388] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:35:58.189631 27582 raft_consensus.cc:482] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:35:58.189801 27582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:35:58.192621 27582 raft_consensus.cc:504] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } }
I20260807 19:35:58.193157 27582 leader_election.cc:302] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e292ee3937e741ce82fe710dc81d575b; no voters:
I20260807 19:35:58.193913 27582 leader_election.cc:288] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:35:58.194072 27588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:35:58.195600 27588 raft_consensus.cc:686] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [term 1 LEADER]: Becoming Leader. State: Replica: e292ee3937e741ce82fe710dc81d575b, State: Running, Role: LEADER
I20260807 19:35:58.196381 27582 sys_catalog.cc:564] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:35:58.196301 27588 consensus_queue.cc:237] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } }
I20260807 19:35:58.199383 27590 sys_catalog.cc:455] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e292ee3937e741ce82fe710dc81d575b. Latest consensus state: current_term: 1 leader_uuid: "e292ee3937e741ce82fe710dc81d575b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } } }
I20260807 19:35:58.199503 27589 sys_catalog.cc:455] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e292ee3937e741ce82fe710dc81d575b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e292ee3937e741ce82fe710dc81d575b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 37151 } } }
I20260807 19:35:58.199823 27590 sys_catalog.cc:458] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:58.199892 27589 sys_catalog.cc:458] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b [sys.catalog]: This master's current role is: LEADER
I20260807 19:35:58.203084 27595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:35:58.209043 27595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:35:58.219427 27595 catalog_manager.cc:1347] Generated new cluster ID: df6460bb9bd8471d81b9369a92ea86b4
I20260807 19:35:58.219581 27595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:35:58.230122 27595 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:35:58.231494 27595 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:35:58.239969 27595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: Generated new TSK 0
I20260807 19:35:58.241151 27595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:35:58.323681 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:58.684785 27611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:58.685086 27611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:58.685272 27611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:58.692894 27611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:58.693049 27611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:58.693239 27611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:35:58.703856 27611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:58.705353 27611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:58.706835 27611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:58.720547 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:58.720664 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:58.720690 27617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:58.721737 27611 server_base.cc:1034] running on GCE node
I20260807 19:35:58.722451 27611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:58.723416 27611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:58.724644 27611 hybrid_clock.cc:648] HybridClock initialized: now 1786131358724586 us; error 55 us; skew 500 ppm
I20260807 19:35:58.724967 27611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:58 dist-test-slave-8w9v krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:58.728241 27611 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:35:58.729791 27611 webserver.cc:460] Webserver started at http://127.8.216.129:44919/ using document root <none> and password file <none>
I20260807 19:35:58.730506 27611 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:58.730664 27611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:58.730999 27611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:35:58.734062 27611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "a3a3f03308504a19b33e362cabc0faa6"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "b458c5bbcd0f10ac7e37038da7275104"
server_key_iv: "98a01c7e7275bc11db8971cb93fffcde"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.734884 27611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "a3a3f03308504a19b33e362cabc0faa6"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "b458c5bbcd0f10ac7e37038da7275104"
server_key_iv: "98a01c7e7275bc11db8971cb93fffcde"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.740530 27611 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:35:58.743983 27627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:58.745464 27611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:35:58.745599 27611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "a3a3f03308504a19b33e362cabc0faa6"
format_stamp: "Formatted at 2026-08-07 19:35:58 on dist-test-slave-8w9v"
server_key: "b458c5bbcd0f10ac7e37038da7275104"
server_key_iv: "98a01c7e7275bc11db8971cb93fffcde"
server_key_version: "encryptionkey@0"
I20260807 19:35:58.745777 27611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:58.757858 27611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:58.762481 27611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:58.762784 27611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:58.763815 27611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:58.765200 27611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:58.765275 27611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:58.765419 27611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:58.765480 27611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:58.800413 27611 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:46815
I20260807 19:35:58.800467 27750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:46815 every 8 connection(s)
I20260807 19:35:58.801787 27611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:35:58.806828 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27611
I20260807 19:35:58.807054 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:35:58.807934 9058 external_mini_cluster.cc:1428] Setting key 9e72ef91e7253a86541d29a78d0d7b2e
Aug 07 19:35:58 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131358, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:58.825387 27753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:58.804571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:33405 (local address 127.8.216.190:37151)
0807 19:35:58.805066 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:58.805071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:58.806178 (+ 1107us) server_negotiation.cc:407] Connection header received
0807 19:35:58.809073 (+ 2895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:58.809078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:58.809177 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:58.809325 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:58.811059 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.812306 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.813905 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.814663 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.814814 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:58.818915 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:58.818920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:58.818924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:58.818958 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:58.821638 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:58.822628 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:58.822632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:58.822634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:58.822755 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:58.823268 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:58.823275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:58.823281 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:58.823675 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:58.823839 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:58.824612 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 19:35:58.824878 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":219,"threads_started":1}
I20260807 19:35:58.826218 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:58.805256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:37151 (local address 127.8.216.129:33405)
0807 19:35:58.805999 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:58.806038 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:58.808771 (+ 2733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:58.809564 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:58.809573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:58.810253 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:58.810866 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:58.810884 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.812484 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.812488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:58.813746 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:58.813758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:58.814888 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:58.814893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:58.815094 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:58.815833 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:58.815857 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:58.818645 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:58.821791 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:58.821803 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:58.821824 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:58.822504 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:58.822870 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:58.822877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:58.822884 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:58.823148 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:58.823803 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:58.823809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:58.824207 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 19:35:58.824467 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 19:35:58.824877 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":306,"threads_started":1}
I20260807 19:35:58.827957 27751 heartbeater.cc:347] Connected to a master server at 127.8.216.190:37151
I20260807 19:35:58.828322 27751 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:58.829071 27751 heartbeater.cc:511] Master 127.8.216.190:37151 requested a full tablet report, sending...
I20260807 19:35:58.831225 27543 ts_manager.cc:194] Registered new tserver with Master: a3a3f03308504a19b33e362cabc0faa6 (127.8.216.129:46815)
I20260807 19:35:58.832827 27543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:33405
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:35:58.879225 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:59.235888 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:59.236231 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:59.236452 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:59.244061 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:59.244279 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:59.244488 27757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:35:59.254885 27757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:59.256459 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:59.257874 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:59.271786 27762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:59.272079 27763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:59.272576 27765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:59.272861 27757 server_base.cc:1034] running on GCE node
I20260807 19:35:59.273633 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:59.274762 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:59.276026 27757 hybrid_clock.cc:648] HybridClock initialized: now 1786131359275952 us; error 73 us; skew 500 ppm
I20260807 19:35:59.276513 27757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:59 dist-test-slave-8w9v krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:59.280433 27757 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:35:59.282207 27757 webserver.cc:460] Webserver started at http://127.8.216.130:43017/ using document root <none> and password file <none>
I20260807 19:35:59.282964 27757 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:59.283123 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:59.283471 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:35:59.286701 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "c6594f0d183043a4a24471a3692661a0"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "ea15a4be7d91ccdb97eb24dd7c9b791b"
server_key_iv: "62805f9cc6004898719ca124bfff2846"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.287583 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "c6594f0d183043a4a24471a3692661a0"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "ea15a4be7d91ccdb97eb24dd7c9b791b"
server_key_iv: "62805f9cc6004898719ca124bfff2846"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.293602 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:35:59.297508 27772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.298988 27757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:35:59.299191 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "c6594f0d183043a4a24471a3692661a0"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "ea15a4be7d91ccdb97eb24dd7c9b791b"
server_key_iv: "62805f9cc6004898719ca124bfff2846"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.299436 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:59.323318 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:59.328302 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:59.328634 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:59.329656 27757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:59.331127 27757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:59.331235 27757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.331384 27757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:59.331458 27757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.367576 27757 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:40979
I20260807 19:35:59.367627 27890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:40979 every 8 connection(s)
I20260807 19:35:59.369038 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:35:59.372179 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27757
I20260807 19:35:59.372383 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:35:59.372833 9058 external_mini_cluster.cc:1428] Setting key c03f8e9457bbe6f1bdc10ef756b15331
Aug 07 19:35:59 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:59.393940 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:59.371891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:41505 (local address 127.8.216.190:37151)
0807 19:35:59.372469 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:59.372476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:59.373691 (+ 1215us) server_negotiation.cc:407] Connection header received
0807 19:35:59.376107 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:59.376111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:59.376229 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:59.376322 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:59.378567 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.379682 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.382277 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.382751 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.382862 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:59.387340 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:59.387345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:59.387349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:59.387383 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:59.390315 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.391327 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.391331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.391333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.391487 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.391997 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.392000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.392002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.392350 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:59.392471 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:59.393428 (+ 957us) server_negotiation.cc:299] Negotiation successful
0807 19:35:59.393640 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":279,"threads_started":1}
I20260807 19:35:59.395653 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:59.372511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:37151 (local address 127.8.216.130:41505)
0807 19:35:59.373420 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:59.373475 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:59.375759 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:59.376625 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:59.376639 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:59.377503 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:59.378358 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:59.378392 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.379847 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.379853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:59.382071 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:59.382088 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.383003 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.383013 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:59.383368 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:59.384182 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:59.384219 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:59.387063 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:59.390483 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:59.390495 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:59.390517 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:59.391199 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:59.391599 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:59.391606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:59.391612 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:59.391883 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:59.392493 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:59.392505 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:59.393014 (+ 509us) client_negotiation.cc:769] Sending connection context
0807 19:35:59.393345 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 19:35:59.394050 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":342,"threads_started":1}
I20260807 19:35:59.397629 27891 heartbeater.cc:347] Connected to a master server at 127.8.216.190:37151
I20260807 19:35:59.398031 27891 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:59.398840 27891 heartbeater.cc:511] Master 127.8.216.190:37151 requested a full tablet report, sending...
I20260807 19:35:59.400553 27543 ts_manager.cc:194] Registered new tserver with Master: c6594f0d183043a4a24471a3692661a0 (127.8.216.130:40979)
I20260807 19:35:59.401868 27543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:41505
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:35:59.448081 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:35:59.815645 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:35:59.815949 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:35:59.816213 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:35:59.823773 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:35:59.823899 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:35:59.824080 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:35:59.834242 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:37151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:35:59.835628 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:35:59.835655 27751 heartbeater.cc:503] Master 127.8.216.190:37151 was elected leader, sending a full tablet report...
I20260807 19:35:59.837479 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:35:59.853274 27903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:59.853377 27905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:35:59.853399 27907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:35:59.854460 27898 server_base.cc:1034] running on GCE node
I20260807 19:35:59.855104 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:35:59.856236 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:35:59.857477 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786131359857429 us; error 57 us; skew 500 ppm
I20260807 19:35:59.857823 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:35:59 dist-test-slave-8w9v krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:35:59.861284 27898 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:35:59.863045 27898 webserver.cc:460] Webserver started at http://127.8.216.131:43031/ using document root <none> and password file <none>
I20260807 19:35:59.863821 27898 fs_manager.cc:359] Metadata directory not provided
I20260807 19:35:59.863981 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:35:59.864405 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:35:59.867782 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "d44d5d558978461a9a4aee6885b6c99a"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "d93431f22cb86ba448d71811997177cb"
server_key_iv: "f74b582f22e0ddd8b036210383c6838a"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.868731 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "d44d5d558978461a9a4aee6885b6c99a"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "d93431f22cb86ba448d71811997177cb"
server_key_iv: "f74b582f22e0ddd8b036210383c6838a"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.874203 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:35:59.877947 27916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.879303 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:35:59.879515 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "d44d5d558978461a9a4aee6885b6c99a"
format_stamp: "Formatted at 2026-08-07 19:35:59 on dist-test-slave-8w9v"
server_key: "d93431f22cb86ba448d71811997177cb"
server_key_iv: "f74b582f22e0ddd8b036210383c6838a"
server_key_version: "encryptionkey@0"
I20260807 19:35:59.879752 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:35:59.893952 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:35:59.898777 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:35:59.899106 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:35:59.900110 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:35:59.901623 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:35:59.901703 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.901880 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:35:59.901955 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:35:59.935729 27898 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:35827
I20260807 19:35:59.935777 28036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:35827 every 8 connection(s)
I20260807 19:35:59.937191 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:35:59.942018 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 27898
I20260807 19:35:59.942220 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:35:59.942809 9058 external_mini_cluster.cc:1428] Setting key f31e1bd80692418e62fd323bb35b5de1
Aug 07 19:35:59 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131359, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:35:59.958801 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:59.939894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:60843 (local address 127.8.216.190:37151)
0807 19:35:59.940445 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:59.940451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:59.941437 (+ 986us) server_negotiation.cc:407] Connection header received
0807 19:35:59.943920 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:59.943925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:59.944022 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:59.944166 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:59.945749 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.946563 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.948199 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.948593 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.948728 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:59.952484 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:59.952489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:59.952492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:59.952524 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:59.955032 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.955953 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.955976 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.955979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.956121 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.956633 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.956638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.956641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.957024 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:59.957189 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:59.958209 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0807 19:35:59.958487 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
I20260807 19:35:59.959824 28041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:59.940542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:37151 (local address 127.8.216.131:60843)
0807 19:35:59.941263 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 19:35:59.941306 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:35:59.943635 (+ 2329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:35:59.944435 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:35:59.944444 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:35:59.944992 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:35:59.945576 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:59.945592 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.946736 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.946742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:59.948004 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:35:59.948016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.948845 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.948853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:35:59.949078 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:59.949850 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:35:59.949873 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:35:59.952271 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:35:59.955361 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:59.955373 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:59.955386 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:59.955822 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:59.956300 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:35:59.956305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:35:59.956309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:35:59.956508 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:35:59.957327 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:35:59.957334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:35:59.957782 (+ 448us) client_negotiation.cc:769] Sending connection context
0807 19:35:59.958042 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 19:35:59.958609 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":290,"threads_started":1}
I20260807 19:35:59.961318 28037 heartbeater.cc:347] Connected to a master server at 127.8.216.190:37151
I20260807 19:35:59.961645 28037 heartbeater.cc:464] Registering TS with master...
I20260807 19:35:59.962411 28037 heartbeater.cc:511] Master 127.8.216.190:37151 requested a full tablet report, sending...
I20260807 19:35:59.963971 27543 ts_manager.cc:194] Registered new tserver with Master: d44d5d558978461a9a4aee6885b6c99a (127.8.216.131:35827)
I20260807 19:35:59.964843 27543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:60843
I20260807 19:35:59.970885 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:35:59.988662 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:35:59.975608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.8.216.190:37151)
0807 19:35:59.975775 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:35:59.975779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:35:59.976342 (+ 563us) server_negotiation.cc:407] Connection header received
0807 19:35:59.976519 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:35:59.976523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:35:59.976591 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:35:59.976678 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:35:59.978169 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.978915 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.980320 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:35:59.980731 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:35:59.980825 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:35:59.983273 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:35:59.983284 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:35:59.983288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:35:59.983322 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:35:59.986155 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.986814 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.986818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.986820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.986940 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:35:59.987277 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:35:59.987280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:35:59.987282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:35:59.987612 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:35:59.987790 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:35:59.988219 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 19:35:59.988398 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:35:59.992875 27543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:00.000054 27543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:00.040004 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.014333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.8.216.129:46815)
0807 19:36:00.030691 (+ 16358us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.030704 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.030741 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:36:00.030880 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.030889 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.031217 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.032392 (+ 1175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:00.032663 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034415 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.037529 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.039010 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.039153 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.039385 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.039629 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.039758 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16162,"thread_start_us":15992,"threads_started":1}
I20260807 19:36:00.039994 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.015375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:40979 (local address 127.0.0.1:55128)
0807 19:36:00.029747 (+ 14372us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:00.029775 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:00.029950 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:00.032284 (+ 2334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:00.032289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:00.032320 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:00.032875 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.032888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034540 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.034547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.037393 (+ 2846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.037406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.038797 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.038803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.039024 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.039119 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 19:36:00.039279 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 19:36:00.039491 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14194,"mutex_wait_us":91,"thread_start_us":14038,"threads_started":1}
I20260807 19:36:00.040115 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.013802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:46815 (local address 127.0.0.1:51148)
0807 19:36:00.014967 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:00.015004 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:00.015175 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:00.031560 (+ 16385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:00.031569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:00.031607 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:00.032145 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.032157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034626 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.034636 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.037286 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.037299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.039238 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.039247 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.039446 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.039478 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:36:00.039568 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 19:36:00.039706 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":89,"thread_start_us":739,"threads_started":1}
I20260807 19:36:00.041224 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.014301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:35827 (local address 127.0.0.1:35830)
0807 19:36:00.028271 (+ 13970us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:00.028302 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:00.028525 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:00.032140 (+ 3615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:00.032146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:00.032174 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:00.032701 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.032716 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034708 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.034714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.037243 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.037266 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.040336 (+ 3070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.040344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.040548 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.040580 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:36:00.040663 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 19:36:00.040802 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13717,"mutex_wait_us":1,"thread_start_us":13512,"threads_started":1}
I20260807 19:36:00.041263 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.015739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55128 (local address 127.8.216.130:40979)
0807 19:36:00.031630 (+ 15891us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.031638 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.031668 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:36:00.031768 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.031775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.032009 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.033079 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:00.033341 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034373 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.037543 (+ 3170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.038557 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.038727 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.038970 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.040713 (+ 1743us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.040914 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15700,"thread_start_us":15546,"threads_started":1}
I20260807 19:36:00.043440 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.015805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.8.216.131:35827)
0807 19:36:00.031157 (+ 15352us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.031165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.031201 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:36:00.031302 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.031309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.031563 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.031782 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:00.032982 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.034573 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.037452 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.039010 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.039153 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.039385 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.042879 (+ 3494us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.043077 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15154,"thread_start_us":15008,"threads_started":1}
I20260807 19:36:00.044782 27823 tablet_service.cc:1467] Processing CreateTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 (DEFAULT_TABLE table=test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:00.044782 27678 tablet_service.cc:1467] Processing CreateTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 (DEFAULT_TABLE table=test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:00.046381 27678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657852b39704e61b98217eb9e6fd6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:00.046392 27823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657852b39704e61b98217eb9e6fd6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:00.047457 27968 tablet_service.cc:1467] Processing CreateTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 (DEFAULT_TABLE table=test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:00.049052 27968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4657852b39704e61b98217eb9e6fd6c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:00.058866 28058 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Bootstrap starting.
I20260807 19:36:00.063575 28058 tablet_bootstrap.cc:654] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:00.064100 28059 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Bootstrap starting.
I20260807 19:36:00.065233 28058 log.cc:824] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:00.066841 28060 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Bootstrap starting.
I20260807 19:36:00.068305 28059 tablet_bootstrap.cc:654] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:00.068809 28058 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: No bootstrap required, opened a new log
I20260807 19:36:00.069149 28058 ts_tablet_manager.cc:1397] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:36:00.069759 28059 log.cc:824] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:00.071091 28060 tablet_bootstrap.cc:654] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:00.072482 28060 log.cc:824] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:00.073419 28059 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: No bootstrap required, opened a new log
I20260807 19:36:00.073738 28059 ts_tablet_manager.cc:1397] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260807 19:36:00.076154 28060 tablet_bootstrap.cc:492] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: No bootstrap required, opened a new log
I20260807 19:36:00.076454 28060 ts_tablet_manager.cc:1397] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 19:36:00.078588 28058 raft_consensus.cc:348] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.079033 28058 raft_consensus.cc:374] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:00.079183 28058 raft_consensus.cc:729] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a3f03308504a19b33e362cabc0faa6, State: Initialized, Role: FOLLOWER
I20260807 19:36:00.079883 28058 consensus_queue.cc:260] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.084796 28058 ts_tablet_manager.cc:1428] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 19:36:00.084977 28059 raft_consensus.cc:348] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.085558 28059 raft_consensus.cc:374] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:00.085673 28059 raft_consensus.cc:729] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6594f0d183043a4a24471a3692661a0, State: Initialized, Role: FOLLOWER
I20260807 19:36:00.086355 28059 consensus_queue.cc:260] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.087611 27891 heartbeater.cc:503] Master 127.8.216.190:37151 was elected leader, sending a full tablet report...
I20260807 19:36:00.087641 28060 raft_consensus.cc:348] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.088158 28060 raft_consensus.cc:374] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:00.088270 28060 raft_consensus.cc:729] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d44d5d558978461a9a4aee6885b6c99a, State: Initialized, Role: FOLLOWER
I20260807 19:36:00.088388 28059 ts_tablet_manager.cc:1428] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
I20260807 19:36:00.089069 28060 consensus_queue.cc:260] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.090281 28037 heartbeater.cc:503] Master 127.8.216.190:37151 was elected leader, sending a full tablet report...
I20260807 19:36:00.091028 28060 ts_tablet_manager.cc:1428] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260807 19:36:00.121198 27893 tablet.cc:2369] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:00.188920 28039 tablet.cc:2369] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:00.304494 27752 tablet.cc:2369] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:00.428141 28064 raft_consensus.cc:482] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:00.428514 28064 raft_consensus.cc:504] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.430385 28064 leader_election.cc:288] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6594f0d183043a4a24471a3692661a0 (127.8.216.130:40979), d44d5d558978461a9a4aee6885b6c99a (127.8.216.131:35827)
I20260807 19:36:00.437840 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.431096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:39873 (local address 127.8.216.131:35827)
0807 19:36:00.431349 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.431358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.432151 (+ 793us) server_negotiation.cc:407] Connection header received
0807 19:36:00.432375 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.432379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.432435 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.432529 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:00.433239 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.433962 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.435928 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.436857 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.437050 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.437132 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.437418 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.437548 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:36:00.437888 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.430823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:43347 (local address 127.8.216.130:40979)
0807 19:36:00.431059 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.431067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.432087 (+ 1020us) server_negotiation.cc:407] Connection header received
0807 19:36:00.432359 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.432363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.432426 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.432532 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:00.433165 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.433903 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.435993 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.436879 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.437063 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.437151 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.437473 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.437606 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:36:00.438037 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.430986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:35827 (local address 127.8.216.129:39873)
0807 19:36:00.432006 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:00.432021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:00.432216 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:00.432556 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:00.432560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:00.432648 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:00.433065 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.433073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.434115 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.434121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.435746 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.435767 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.437093 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.437101 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.437253 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.437278 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:36:00.437392 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:36:00.437534 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":28,"spinlock_wait_cycles":23424,"thread_start_us":657,"threads_started":1}
I20260807 19:36:00.438473 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.430869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:40979 (local address 127.8.216.129:43347)
0807 19:36:00.431946 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:00.431973 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:00.432180 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:00.432551 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:00.432556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:00.432582 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:00.433005 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.433014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.434049 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.434057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.435801 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:00.435814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.437145 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.437151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:00.437299 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.437322 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:36:00.437436 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:36:00.437569 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":75,"thread_start_us":619,"threads_started":1}
I20260807 19:36:00.439183 27989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657852b39704e61b98217eb9e6fd6c5" candidate_uuid: "a3a3f03308504a19b33e362cabc0faa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d44d5d558978461a9a4aee6885b6c99a" is_pre_election: true
I20260807 19:36:00.439541 27843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657852b39704e61b98217eb9e6fd6c5" candidate_uuid: "a3a3f03308504a19b33e362cabc0faa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6594f0d183043a4a24471a3692661a0" is_pre_election: true
I20260807 19:36:00.439688 27989 raft_consensus.cc:2454] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a3f03308504a19b33e362cabc0faa6 in term 0.
I20260807 19:36:00.440055 27843 raft_consensus.cc:2454] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a3f03308504a19b33e362cabc0faa6 in term 0.
I20260807 19:36:00.440460 27630 leader_election.cc:302] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a3f03308504a19b33e362cabc0faa6, d44d5d558978461a9a4aee6885b6c99a; no voters:
I20260807 19:36:00.440936 28064 raft_consensus.cc:2781] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:00.441105 28064 raft_consensus.cc:482] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:00.441222 28064 raft_consensus.cc:3037] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:00.443517 28064 raft_consensus.cc:504] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.444140 28064 leader_election.cc:288] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [CANDIDATE]: Term 1 election: Requested vote from peers c6594f0d183043a4a24471a3692661a0 (127.8.216.130:40979), d44d5d558978461a9a4aee6885b6c99a (127.8.216.131:35827)
I20260807 19:36:00.444538 27843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657852b39704e61b98217eb9e6fd6c5" candidate_uuid: "a3a3f03308504a19b33e362cabc0faa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6594f0d183043a4a24471a3692661a0"
I20260807 19:36:00.444633 27989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4657852b39704e61b98217eb9e6fd6c5" candidate_uuid: "a3a3f03308504a19b33e362cabc0faa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d44d5d558978461a9a4aee6885b6c99a"
I20260807 19:36:00.444754 27843 raft_consensus.cc:3037] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:00.444900 27989 raft_consensus.cc:3037] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:00.447309 27989 raft_consensus.cc:2454] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a3f03308504a19b33e362cabc0faa6 in term 1.
I20260807 19:36:00.447319 27843 raft_consensus.cc:2454] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a3f03308504a19b33e362cabc0faa6 in term 1.
I20260807 19:36:00.447769 27630 leader_election.cc:302] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a3f03308504a19b33e362cabc0faa6, d44d5d558978461a9a4aee6885b6c99a; no voters:
I20260807 19:36:00.448108 28064 raft_consensus.cc:2781] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:00.448685 28064 raft_consensus.cc:686] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 1 LEADER]: Becoming Leader. State: Replica: a3a3f03308504a19b33e362cabc0faa6, State: Running, Role: LEADER
I20260807 19:36:00.449293 28064 consensus_queue.cc:237] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } }
I20260807 19:36:00.454314 27543 catalog_manager.cc:5495] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3a3f03308504a19b33e362cabc0faa6 (127.8.216.129). New cstate: current_term: 1 leader_uuid: "a3a3f03308504a19b33e362cabc0faa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a3f03308504a19b33e362cabc0faa6" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 46815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260807 19:36:00.499346 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.493118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.8.216.129:46815)
0807 19:36:00.493313 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.493318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.493765 (+ 447us) server_negotiation.cc:407] Connection header received
0807 19:36:00.493962 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.493966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.494072 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.494206 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:00.494609 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.495309 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.497020 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.497536 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.497640 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.498219 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:00.498778 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:00.498903 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.499033 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.499109 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:36:00.508231 27843 raft_consensus.cc:1261] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 1 FOLLOWER]: Refusing update from remote peer a3a3f03308504a19b33e362cabc0faa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:00.508373 27989 raft_consensus.cc:1261] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 1 FOLLOWER]: Refusing update from remote peer a3a3f03308504a19b33e362cabc0faa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:00.509568 28069 consensus_queue.cc:1035] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6594f0d183043a4a24471a3692661a0" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 40979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:00.510057 28064 consensus_queue.cc:1035] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d44d5d558978461a9a4aee6885b6c99a" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:00.522411 28074 mvcc.cc:205] Tried to move back new op lower bound from 7315994052630646784 to 7315994052404830208. Current Snapshot: MvccSnapshot[applied={T|T < 7315994052630646784}]
I20260807 19:36:00.536402 27543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:00.536684 27543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:00.540568 27543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e292ee3937e741ce82fe710dc81d575b: Sending DeleteTablet for 3 replicas of tablet 4657852b39704e61b98217eb9e6fd6c5
I20260807 19:36:00.541880 27678 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:51148
I20260807 19:36:00.541962 27823 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:55128
I20260807 19:36:00.541983 27968 tablet_service.cc:1514] Processing DeleteTablet for tablet 4657852b39704e61b98217eb9e6fd6c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:00 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:35830
I20260807 19:36:00.550288 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27611
I20260807 19:36:00.553825 28085 tablet_replica.cc:330] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: stopping tablet replica
I20260807 19:36:00.554466 28085 raft_consensus.cc:2229] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:00.555171 28085 raft_consensus.cc:2258] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:00.558020 28084 tablet_replica.cc:330] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: stopping tablet replica
I20260807 19:36:00.558568 28084 raft_consensus.cc:2229] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:00.559437 28084 raft_consensus.cc:2258] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:00.560138 28083 tablet_replica.cc:330] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: stopping tablet replica
I20260807 19:36:00.560632 28083 raft_consensus.cc:2229] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:00.561236 28083 raft_consensus.cc:2258] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:00.561690 28085 ts_tablet_manager.cc:1905] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:00.562019 28084 ts_tablet_manager.cc:1905] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:36:00 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:36:00.569777 28085 ts_tablet_manager.cc:1918] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:00.570132 28085 log.cc:1196] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/4657852b39704e61b98217eb9e6fd6c5
I20260807 19:36:00.570288 28084 ts_tablet_manager.cc:1918] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:00.570477 28084 log.cc:1196] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/4657852b39704e61b98217eb9e6fd6c5
I20260807 19:36:00.571036 28085 ts_tablet_manager.cc:1939] T 4657852b39704e61b98217eb9e6fd6c5 P d44d5d558978461a9a4aee6885b6c99a: Deleting consensus metadata
I20260807 19:36:00.571049 28084 ts_tablet_manager.cc:1939] T 4657852b39704e61b98217eb9e6fd6c5 P a3a3f03308504a19b33e362cabc0faa6: Deleting consensus metadata
I20260807 19:36:00.573167 27528 catalog_manager.cc:4841] TS d44d5d558978461a9a4aee6885b6c99a (127.8.216.131:35827): tablet 4657852b39704e61b98217eb9e6fd6c5 (table test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]) successfully deleted
I20260807 19:36:00.573931 27529 catalog_manager.cc:4841] TS a3a3f03308504a19b33e362cabc0faa6 (127.8.216.129:46815): tablet 4657852b39704e61b98217eb9e6fd6c5 (table test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]) successfully deleted
I20260807 19:36:00.574532 28083 ts_tablet_manager.cc:1905] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:00.575847 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.551618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.8.216.129:46815)
0807 19:36:00.551797 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.551804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.552011 (+ 207us) server_negotiation.cc:407] Connection header received
0807 19:36:00.552158 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.552165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.552257 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.552376 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:00.555490 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.556806 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.558883 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.559651 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.559763 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.565852 (+ 6089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:00.565869 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:00.565890 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:00.565942 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:00.570857 (+ 4915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.572061 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.572071 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.572074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.572280 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.573510 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.573517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.573520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.574077 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:00.574362 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.574973 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.575246 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:36:00.576704 27745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:00.582479 28083 ts_tablet_manager.cc:1918] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:00.582677 28083 log.cc:1196] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/4657852b39704e61b98217eb9e6fd6c5
I20260807 19:36:00.583304 28083 ts_tablet_manager.cc:1939] T 4657852b39704e61b98217eb9e6fd6c5 P c6594f0d183043a4a24471a3692661a0: Deleting consensus metadata
I20260807 19:36:00.585533 27529 catalog_manager.cc:4841] TS c6594f0d183043a4a24471a3692661a0 (127.8.216.130:40979): tablet 4657852b39704e61b98217eb9e6fd6c5 (table test-table [id=8c5ff974684d42f0b8e7de0decd4fe6c]) successfully deleted
I20260807 19:36:00.682142 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27611
I20260807 19:36:00.704835 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27757
Aug 07 19:36:00 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:36:00.720506 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.705709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.8.216.130:40979)
0807 19:36:00.705963 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.705968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.705987 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:36:00.706118 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.706122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.706187 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.706271 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:00.707473 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.708276 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.709641 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.710058 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.710153 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.714631 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:00.714650 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:00.714667 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:00.714711 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:00.717956 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.718687 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.718693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.718696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.718824 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.719140 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.719143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.719145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.719526 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:00.719759 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.719980 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.720226 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 19:36:00.721194 27886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:00.815102 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27757
I20260807 19:36:00.836795 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27898
Aug 07 19:36:00 dist-test-slave-8w9v krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:36:00.852907 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:00.837705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.8.216.131:35827)
0807 19:36:00.837898 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:00.837903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:00.837922 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:36:00.837982 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:00.837986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:00.838076 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:00.838165 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:00.839632 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.840493 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.841791 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:00.842172 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:00.842269 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:00.846900 (+ 4631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:00.846919 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:00.846936 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:00.846974 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:00.849830 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.850593 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.850601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.850603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.850749 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:00.851159 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:00.851165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:00.851168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:00.851577 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:00.851892 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:00.852271 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 19:36:00.852532 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:36:00.853616 28031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:00.946833 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27898
I20260807 19:36:00.968108 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 27508
I20260807 19:36:00.969580 27577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:01.085031 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 27508
2026-08-07T19:36:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4977 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T19:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:01Z Disabled control of system clock
I20260807 19:36:01.188469 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38049
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:38049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:01.546142 28096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:01.546476 28096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:01.546630 28096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:01.554234 28096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:01.554387 28096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:01.554478 28096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:01.554589 28096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:01.554673 28096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:01.564513 28096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:38049
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:01.565927 28096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:01.567865 28096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:01.580971 28102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:01.581018 28105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:01.581070 28103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:01.585750 28096 server_base.cc:1034] running on GCE node
I20260807 19:36:01.586644 28096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:01.588322 28096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:01.589573 28096 hybrid_clock.cc:648] HybridClock initialized: now 1786131361589532 us; error 58 us; skew 500 ppm
I20260807 19:36:01.589958 28096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:01.592757 28096 webserver.cc:460] Webserver started at http://127.8.216.190:45619/ using document root <none> and password file <none>
I20260807 19:36:01.593706 28096 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:01.593855 28096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:01.594208 28096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:01.597434 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "1ece79d5312d4c4e9ba2f178d18ad585"
format_stamp: "Formatted at 2026-08-07 19:36:01 on dist-test-slave-8w9v"
server_key: "613443b679fcd445e941e4de2ab1cb76"
server_key_iv: "aba864ddd34bf272c873af9a567b5be9"
server_key_version: "encryptionkey@0"
I20260807 19:36:01.598315 28096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "1ece79d5312d4c4e9ba2f178d18ad585"
format_stamp: "Formatted at 2026-08-07 19:36:01 on dist-test-slave-8w9v"
server_key: "613443b679fcd445e941e4de2ab1cb76"
server_key_iv: "aba864ddd34bf272c873af9a567b5be9"
server_key_version: "encryptionkey@0"
I20260807 19:36:01.604003 28096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:36:01.607736 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:01.609210 28096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:01.609449 28096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "1ece79d5312d4c4e9ba2f178d18ad585"
format_stamp: "Formatted at 2026-08-07 19:36:01 on dist-test-slave-8w9v"
server_key: "613443b679fcd445e941e4de2ab1cb76"
server_key_iv: "aba864ddd34bf272c873af9a567b5be9"
server_key_version: "encryptionkey@0"
I20260807 19:36:01.609721 28096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:01.645274 28096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:01.646150 28096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:01.646435 28096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:01.667290 28096 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:38049
I20260807 19:36:01.667310 28168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:38049 every 8 connection(s)
I20260807 19:36:01.668779 28096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:01.670986 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28096
I20260807 19:36:01.671169 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:01.671655 9058 external_mini_cluster.cc:1428] Setting key 4b1e699c53d6fe6fc36bcef4009be15c
I20260807 19:36:01.673099 28169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:01.686115 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:01.673736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53702 (local address 127.8.216.190:38049)
0807 19:36:01.674833 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:01.674845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:01.674886 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:36:01.676562 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:01.676584 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:01.677082 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:01.677438 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:01.678085 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:01.679459 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:01.681405 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:01.682064 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:01.682196 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:01.683557 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:01.683567 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:01.683581 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:01.684227 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:01.684318 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:01.684346 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:01.684450 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:01.684730 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:36:01.685116 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":448,"threads_started":1}
I20260807 19:36:01.687676 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: Bootstrap starting.
I20260807 19:36:01.691272 28169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:01.692376 28169 log.cc:824] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:01.695475 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: No bootstrap required, opened a new log
I20260807 19:36:01.703428 28169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } }
I20260807 19:36:01.703840 28169 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:01.703999 28169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ece79d5312d4c4e9ba2f178d18ad585, State: Initialized, Role: FOLLOWER
I20260807 19:36:01.704618 28169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } }
I20260807 19:36:01.704867 28169 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:01.705017 28169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:01.705217 28169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:01.707558 28169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } }
I20260807 19:36:01.708113 28169 leader_election.cc:302] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ece79d5312d4c4e9ba2f178d18ad585; no voters:
I20260807 19:36:01.708983 28169 leader_election.cc:288] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:01.709124 28175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:01.709892 28175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [term 1 LEADER]: Becoming Leader. State: Replica: 1ece79d5312d4c4e9ba2f178d18ad585, State: Running, Role: LEADER
I20260807 19:36:01.710472 28175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } }
I20260807 19:36:01.711397 28169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:01.713470 28177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ece79d5312d4c4e9ba2f178d18ad585. Latest consensus state: current_term: 1 leader_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } } }
I20260807 19:36:01.713443 28176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ece79d5312d4c4e9ba2f178d18ad585" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38049 } } }
I20260807 19:36:01.713979 28176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:01.714802 28177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:01.715734 28182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:01.721446 28182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:01.731585 28182 catalog_manager.cc:1347] Generated new cluster ID: 78e09d6c195a47deb6a489af8ddae383
I20260807 19:36:01.731721 28182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:01.756557 28182 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:01.758527 28182 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:01.774302 28182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: Generated new TSK 0
I20260807 19:36:01.775458 28182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:01.780652 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--builtin_ntp_servers=127.8.216.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:02.144043 28197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:02.144445 28197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:02.144634 28197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:02.152571 28197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:02.152735 28197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:02.152936 28197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:36:02.162755 28197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:02.164285 28197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:02.166208 28197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:02.181063 28207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:02.181088 28204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:02.181063 28205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:02.182004 28197 server_base.cc:1034] running on GCE node
I20260807 19:36:02.182652 28197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:02.183653 28197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:02.184890 28197 hybrid_clock.cc:648] HybridClock initialized: now 1786131362184841 us; error 59 us; skew 500 ppm
I20260807 19:36:02.185247 28197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:02.187721 28197 webserver.cc:460] Webserver started at http://127.8.216.129:38389/ using document root <none> and password file <none>
I20260807 19:36:02.188552 28197 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:02.188694 28197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:02.189050 28197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:02.192278 28197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "0bacd61cc03d47989c9c03dfb3e0e014"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "cb771f0f22f9b40468a520d1f90609ba"
server_key_iv: "d0ba72eebef135296722eb719d69c5d1"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.193136 28197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "0bacd61cc03d47989c9c03dfb3e0e014"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "cb771f0f22f9b40468a520d1f90609ba"
server_key_iv: "d0ba72eebef135296722eb719d69c5d1"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.199203 28197 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:36:02.203008 28213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.204466 28197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:36:02.204659 28197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "0bacd61cc03d47989c9c03dfb3e0e014"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "cb771f0f22f9b40468a520d1f90609ba"
server_key_iv: "d0ba72eebef135296722eb719d69c5d1"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.204898 28197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:02.236793 28197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:02.237672 28197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:02.237959 28197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:02.239042 28197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:02.240631 28197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:02.240720 28197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.240809 28197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:02.240866 28197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.277055 28197 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:34843
I20260807 19:36:02.277082 28341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:34843 every 8 connection(s)
I20260807 19:36:02.278537 28197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:36:02.286870 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28197
I20260807 19:36:02.287074 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:36:02.287569 9058 external_mini_cluster.cc:1428] Setting key e15d352508d39e2e428f0afbd32c2390
I20260807 19:36:02.290376 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--builtin_ntp_servers=127.8.216.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:02.292992 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:02.281579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:58869 (local address 127.8.216.190:38049)
0807 19:36:02.282057 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:02.282061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:02.283484 (+ 1423us) server_negotiation.cc:407] Connection header received
0807 19:36:02.285594 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:02.285599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:02.285681 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:02.285788 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:02.287091 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.287990 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.289762 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.290184 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.290347 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:02.292032 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:02.292037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:02.292040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:02.292059 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:02.292099 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:02.292112 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:02.292242 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:02.292511 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 19:36:02.292618 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":258,"threads_started":1}
I20260807 19:36:02.294085 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:02.282233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38049 (local address 127.8.216.129:58869)
0807 19:36:02.283261 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:02.283303 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:02.285307 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:02.286059 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:02.286067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:02.286162 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:02.286874 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:02.286894 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.288238 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.288242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:02.289569 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:02.289584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.290451 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.290459 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:02.290823 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:02.291655 (+ 832us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:02.291658 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:02.291709 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:02.291800 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:02.292285 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:02.292296 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:02.292302 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:36:02.292501 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 19:36:02.292931 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":325,"threads_started":1}
I20260807 19:36:02.295907 28342 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38049
I20260807 19:36:02.296317 28342 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:02.297130 28342 heartbeater.cc:511] Master 127.8.216.190:38049 requested a full tablet report, sending...
I20260807 19:36:02.299557 28128 ts_manager.cc:194] Registered new tserver with Master: 0bacd61cc03d47989c9c03dfb3e0e014 (127.8.216.129:34843)
I20260807 19:36:02.301823 28128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.129:58869
W20260807 19:36:02.651151 28346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:02.651517 28346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:02.651666 28346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:02.659269 28346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:02.659387 28346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:02.659567 28346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:36:02.671500 28346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:02.673135 28346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:02.675470 28346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:02.691479 28351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:02.691511 28354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:02.691398 28352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:02.692350 28346 server_base.cc:1034] running on GCE node
I20260807 19:36:02.693073 28346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:02.694182 28346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:02.695478 28346 hybrid_clock.cc:648] HybridClock initialized: now 1786131362695423 us; error 43 us; skew 500 ppm
I20260807 19:36:02.695878 28346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:02.698415 28346 webserver.cc:460] Webserver started at http://127.8.216.130:45927/ using document root <none> and password file <none>
I20260807 19:36:02.699198 28346 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:02.699334 28346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:02.699689 28346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:02.702783 28346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "0020cb5beb694100afe3acf0f2a14aa8"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "ca7758db1ac5bd8d03812ccf578010b3"
server_key_iv: "38d6c0225e5c20a9d1f8827fc9c73d03"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.703641 28346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "0020cb5beb694100afe3acf0f2a14aa8"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "ca7758db1ac5bd8d03812ccf578010b3"
server_key_iv: "38d6c0225e5c20a9d1f8827fc9c73d03"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.709363 28346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:36:02.713333 28361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.714892 28346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:36:02.715061 28346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "0020cb5beb694100afe3acf0f2a14aa8"
format_stamp: "Formatted at 2026-08-07 19:36:02 on dist-test-slave-8w9v"
server_key: "ca7758db1ac5bd8d03812ccf578010b3"
server_key_iv: "38d6c0225e5c20a9d1f8827fc9c73d03"
server_key_version: "encryptionkey@0"
I20260807 19:36:02.715260 28346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:02.740201 28346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:02.741048 28346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:02.741320 28346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:02.742286 28346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:02.743817 28346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:02.743907 28346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.744012 28346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:02.744151 28346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:02.778939 28346 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38279
I20260807 19:36:02.779054 28482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38279 every 8 connection(s)
I20260807 19:36:02.780422 28346 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:36:02.786324 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28346
I20260807 19:36:02.786525 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:36:02.787156 9058 external_mini_cluster.cc:1428] Setting key e05d72f130ef97a729ab06e57daa3a99
I20260807 19:36:02.790331 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--builtin_ntp_servers=127.8.216.148:40127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:02.794692 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:02.783132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:55789 (local address 127.8.216.190:38049)
0807 19:36:02.783366 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:02.783374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:02.784818 (+ 1444us) server_negotiation.cc:407] Connection header received
0807 19:36:02.786820 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:02.786827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:02.786937 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:02.787065 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:02.788485 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.789385 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.791171 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.791960 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.792115 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:02.793638 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:02.793648 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:02.793656 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:02.793684 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:02.793731 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:02.793749 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:02.793847 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:02.794104 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 19:36:02.794249 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:36:02.795876 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:02.783662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38049 (local address 127.8.216.130:55789)
0807 19:36:02.784650 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:02.784692 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:02.786474 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:02.787306 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:02.787314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:02.787428 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:02.788252 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:02.788274 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.789553 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.789557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:02.791010 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:02.791024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:02.792255 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:02.792260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:02.792489 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:02.793255 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:02.793258 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:02.793303 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:02.793396 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:02.793853 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:02.793861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:02.793867 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:36:02.794043 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 19:36:02.794442 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":279,"threads_started":1}
I20260807 19:36:02.797552 28483 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38049
I20260807 19:36:02.797892 28483 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:02.798753 28483 heartbeater.cc:511] Master 127.8.216.190:38049 requested a full tablet report, sending...
I20260807 19:36:02.800379 28128 ts_manager.cc:194] Registered new tserver with Master: 0020cb5beb694100afe3acf0f2a14aa8 (127.8.216.130:38279)
I20260807 19:36:02.801218 28128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.130:55789
W20260807 19:36:03.146008 28486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:03.146297 28486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:03.146502 28486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:03.153944 28486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:03.154054 28486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:03.154201 28486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:36:03.163715 28486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:38049
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:03.165146 28486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:03.166991 28486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:03.181066 28492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:03.181270 28491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:03.181252 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:03.185592 28486 server_base.cc:1034] running on GCE node
I20260807 19:36:03.186206 28486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:03.187181 28486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:03.188428 28486 hybrid_clock.cc:648] HybridClock initialized: now 1786131363188377 us; error 63 us; skew 500 ppm
I20260807 19:36:03.188773 28486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:03.194176 28486 webserver.cc:460] Webserver started at http://127.8.216.131:36743/ using document root <none> and password file <none>
I20260807 19:36:03.195040 28486 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:03.195183 28486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:03.195542 28486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:03.198734 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "a7a1b01260144d4abaa3e88f2c902f5e"
format_stamp: "Formatted at 2026-08-07 19:36:03 on dist-test-slave-8w9v"
server_key: "ae49faf7ae4c05da20507ea5d3a27ad7"
server_key_iv: "8a598309f8aa69bc5844f268421294ad"
server_key_version: "encryptionkey@0"
I20260807 19:36:03.199559 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "a7a1b01260144d4abaa3e88f2c902f5e"
format_stamp: "Formatted at 2026-08-07 19:36:03 on dist-test-slave-8w9v"
server_key: "ae49faf7ae4c05da20507ea5d3a27ad7"
server_key_iv: "8a598309f8aa69bc5844f268421294ad"
server_key_version: "encryptionkey@0"
I20260807 19:36:03.205163 28486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 19:36:03.208950 28500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:03.210386 28486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:03.210597 28486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "a7a1b01260144d4abaa3e88f2c902f5e"
format_stamp: "Formatted at 2026-08-07 19:36:03 on dist-test-slave-8w9v"
server_key: "ae49faf7ae4c05da20507ea5d3a27ad7"
server_key_iv: "8a598309f8aa69bc5844f268421294ad"
server_key_version: "encryptionkey@0"
I20260807 19:36:03.210844 28486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:03.245105 28486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:03.246057 28486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:03.246394 28486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:03.247495 28486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:03.249109 28486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:03.249245 28486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:03.249392 28486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:03.249493 28486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:03.285280 28486 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:40747
I20260807 19:36:03.285374 28625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:40747 every 8 connection(s)
I20260807 19:36:03.286754 28486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:36:03.295791 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28486
I20260807 19:36:03.296005 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:36:03.296636 9058 external_mini_cluster.cc:1428] Setting key 8463d0dd84662ff00a7a548ff98850fd
I20260807 19:36:03.300736 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.289469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:38611 (local address 127.8.216.190:38049)
0807 19:36:03.289681 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.289686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.291131 (+ 1445us) server_negotiation.cc:407] Connection header received
0807 19:36:03.293082 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.293087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.293165 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.293268 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:03.294404 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.295170 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.297566 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.298197 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.298320 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.299928 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:03.299933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:03.299936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:03.299990 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:03.300024 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:03.300037 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:03.300161 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.300393 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.300490 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 19:36:03.301749 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.289993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:38049 (local address 127.8.216.131:38611)
0807 19:36:03.290944 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.290990 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.292765 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.293526 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.293534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.293629 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:03.294229 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.294246 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.295361 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.295366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.297376 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.297419 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.298445 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.298455 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.298753 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.299570 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:03.299574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:03.299628 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:03.299719 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:03.300172 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:03.300182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:03.300196 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.300351 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.300654 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":310,"threads_started":1}
I20260807 19:36:03.303289 28626 heartbeater.cc:347] Connected to a master server at 127.8.216.190:38049
I20260807 19:36:03.303664 28626 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:03.304616 28626 heartbeater.cc:511] Master 127.8.216.190:38049 requested a full tablet report, sending...
I20260807 19:36:03.306097 28342 heartbeater.cc:503] Master 127.8.216.190:38049 was elected leader, sending a full tablet report...
I20260807 19:36:03.306110 28128 ts_manager.cc:194] Registered new tserver with Master: a7a1b01260144d4abaa3e88f2c902f5e (127.8.216.131:40747)
I20260807 19:36:03.306949 28128 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.131:38611
I20260807 19:36:03.312289 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:36:03.322938 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.316843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53714 (local address 127.8.216.190:38049)
0807 19:36:03.317044 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.317049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.317438 (+ 389us) server_negotiation.cc:407] Connection header received
0807 19:36:03.317587 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.317591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.317651 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.317733 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:03.318200 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.318988 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.320615 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.321110 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.321262 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.322312 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:03.322319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:03.322326 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:03.322356 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:03.322389 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:03.322401 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:03.322473 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.322605 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.322682 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:36:03.326942 28127 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:03.334122 28127 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:03.371388 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.346998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40747 (local address 127.0.0.1:60016)
0807 19:36:03.357791 (+ 10793us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.359049 (+ 1258us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.359169 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.364311 (+ 5142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.364315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.364339 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:03.364739 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.364753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.366782 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.366790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.368960 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.368972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.370292 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.370299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.370588 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.370642 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.370813 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.371013 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10483,"thread_start_us":16617,"threads_started":1}
I20260807 19:36:03.371407 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.348208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.8.216.131:40747)
0807 19:36:03.363430 (+ 15222us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.363442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.363487 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:36:03.363612 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.363619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.363843 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.364967 (+ 1124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:03.365290 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.366638 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.369157 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.370073 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.370221 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.370539 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.370858 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.371042 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14981,"thread_start_us":15794,"threads_started":1}
I20260807 19:36:03.372540 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.346789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:42316)
0807 19:36:03.357791 (+ 11002us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.357887 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.358157 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.363027 (+ 4870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.363035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.363076 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:03.363551 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.363576 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.365801 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.365807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.368296 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.368311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.370695 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.370702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.371867 (+ 1165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.371901 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.371996 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.372173 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6511,"spinlock_wait_cycles":896,"thread_start_us":589,"threads_started":1}
I20260807 19:36:03.373129 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.347073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.8.216.129:34843)
0807 19:36:03.362167 (+ 15094us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.362182 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.362225 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:36:03.362358 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.362371 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.362754 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.363015 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:03.363822 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.365652 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.368517 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.370128 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.370269 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.370546 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.372291 (+ 1745us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.372490 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14810,"thread_start_us":16152,"threads_started":1}
I20260807 19:36:03.373797 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.353231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:38279 (local address 127.0.0.1:35312)
0807 19:36:03.363945 (+ 10714us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.363964 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.364079 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.366078 (+ 1999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.366083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.366119 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:03.366596 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.366609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.368664 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.368669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.370803 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.370815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.372975 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.372982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.373159 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.373189 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.373286 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.373454 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10557,"mutex_wait_us":83,"thread_start_us":3980,"threads_started":1}
I20260807 19:36:03.374715 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.348187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.8.216.130:38279)
0807 19:36:03.365502 (+ 17315us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.365512 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.365542 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:36:03.365637 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.365646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.365873 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.366064 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:03.366859 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.368508 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.371065 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.372616 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.372768 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.373054 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.374159 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.374356 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17116,"thread_start_us":16973,"threads_started":1}
I20260807 19:36:03.376739 28270 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 (DEFAULT_TABLE table=test-table [id=652e6160ee3d401ca076d561f3ca0615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:03.376868 28554 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 (DEFAULT_TABLE table=test-table [id=652e6160ee3d401ca076d561f3ca0615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:03.378085 28270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab6499e741443ca94c406993ba8ddc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:03.378345 28554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab6499e741443ca94c406993ba8ddc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:03.378499 28414 tablet_service.cc:1467] Processing CreateTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 (DEFAULT_TABLE table=test-table [id=652e6160ee3d401ca076d561f3ca0615]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:03.379773 28414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ab6499e741443ca94c406993ba8ddc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:03.390090 28645 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Bootstrap starting.
I20260807 19:36:03.393158 28646 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Bootstrap starting.
I20260807 19:36:03.394263 28645 tablet_bootstrap.cc:654] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:03.395766 28645 log.cc:824] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:03.396705 28647 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Bootstrap starting.
I20260807 19:36:03.396946 28646 tablet_bootstrap.cc:654] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:03.398447 28646 log.cc:824] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:03.400329 28645 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: No bootstrap required, opened a new log
I20260807 19:36:03.400617 28645 ts_tablet_manager.cc:1397] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260807 19:36:03.400717 28647 tablet_bootstrap.cc:654] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:03.401631 28646 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: No bootstrap required, opened a new log
I20260807 19:36:03.401718 28647 log.cc:824] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:03.401891 28646 ts_tablet_manager.cc:1397] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:36:03.404762 28647 tablet_bootstrap.cc:492] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: No bootstrap required, opened a new log
I20260807 19:36:03.405043 28647 ts_tablet_manager.cc:1397] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 19:36:03.411716 28645 raft_consensus.cc:348] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.412248 28645 raft_consensus.cc:374] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:03.412352 28645 raft_consensus.cc:729] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bacd61cc03d47989c9c03dfb3e0e014, State: Initialized, Role: FOLLOWER
I20260807 19:36:03.412519 28647 raft_consensus.cc:348] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.413000 28647 raft_consensus.cc:374] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:03.413110 28647 raft_consensus.cc:729] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0020cb5beb694100afe3acf0f2a14aa8, State: Initialized, Role: FOLLOWER
I20260807 19:36:03.412935 28646 raft_consensus.cc:348] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.413049 28645 consensus_queue.cc:260] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.413352 28646 raft_consensus.cc:374] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:03.413466 28646 raft_consensus.cc:729] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a1b01260144d4abaa3e88f2c902f5e, State: Initialized, Role: FOLLOWER
I20260807 19:36:03.413754 28647 consensus_queue.cc:260] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.414526 28646 consensus_queue.cc:260] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.417246 28645 ts_tablet_manager.cc:1428] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 19:36:03.418483 28626 heartbeater.cc:503] Master 127.8.216.190:38049 was elected leader, sending a full tablet report...
I20260807 19:36:03.419214 28646 ts_tablet_manager.cc:1428] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260807 19:36:03.420315 28483 heartbeater.cc:503] Master 127.8.216.190:38049 was elected leader, sending a full tablet report...
I20260807 19:36:03.422860 28647 ts_tablet_manager.cc:1428] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Time spent starting tablet: real 0.018s user 0.016s sys 0.001s
W20260807 19:36:03.531500 28343 tablet.cc:2369] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:03.532509 28484 tablet.cc:2369] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:03.538578 28627 tablet.cc:2369] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:03.576946 28653 raft_consensus.cc:482] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:03.577286 28653 raft_consensus.cc:504] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.579276 28653 leader_election.cc:288] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bacd61cc03d47989c9c03dfb3e0e014 (127.8.216.129:34843), a7a1b01260144d4abaa3e88f2c902f5e (127.8.216.131:40747)
I20260807 19:36:03.586403 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.579845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:55401 (local address 127.8.216.129:34843)
0807 19:36:03.580146 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.580157 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.580629 (+ 472us) server_negotiation.cc:407] Connection header received
0807 19:36:03.580881 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.580885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.580964 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.581075 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:03.581735 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.582508 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.584643 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.585448 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.585640 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.585716 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.586010 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.586138 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:36:03.586762 28656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.579693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.8.216.130:55401)
0807 19:36:03.580500 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.580521 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.580719 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.581091 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.581095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.581119 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:03.581524 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.581536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.582663 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.582672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.584433 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.584446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.585656 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.585662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.585823 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.585847 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.585979 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.586125 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":84,"thread_start_us":457,"threads_started":1}
I20260807 19:36:03.586756 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.579905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:37583 (local address 127.8.216.131:40747)
0807 19:36:03.580168 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.580175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.580895 (+ 720us) server_negotiation.cc:407] Connection header received
0807 19:36:03.581137 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.581142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.581210 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.581351 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:03.582015 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.582805 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.584911 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.585743 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.585944 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.586024 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.586365 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.586504 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:36:03.586902 28655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.579698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40747 (local address 127.8.216.130:37583)
0807 19:36:03.580767 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.580783 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:03.580968 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:03.581364 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:03.581369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:03.581395 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:03.581832 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.581842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.582984 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.582993 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.584698 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:03.584719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.585957 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.585963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:03.586148 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.586173 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:36:03.586287 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:36:03.586436 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":77,"thread_start_us":655,"threads_started":1}
I20260807 19:36:03.587978 28293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab6499e741443ca94c406993ba8ddc7" candidate_uuid: "0020cb5beb694100afe3acf0f2a14aa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" is_pre_election: true
I20260807 19:36:03.587988 28575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab6499e741443ca94c406993ba8ddc7" candidate_uuid: "0020cb5beb694100afe3acf0f2a14aa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" is_pre_election: true
I20260807 19:36:03.588577 28293 raft_consensus.cc:2454] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0020cb5beb694100afe3acf0f2a14aa8 in term 0.
I20260807 19:36:03.588596 28575 raft_consensus.cc:2454] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0020cb5beb694100afe3acf0f2a14aa8 in term 0.
I20260807 19:36:03.589329 28362 leader_election.cc:302] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0020cb5beb694100afe3acf0f2a14aa8, 0bacd61cc03d47989c9c03dfb3e0e014; no voters:
I20260807 19:36:03.589793 28653 raft_consensus.cc:2781] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:03.589960 28653 raft_consensus.cc:482] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:03.590080 28653 raft_consensus.cc:3037] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:03.592417 28653 raft_consensus.cc:504] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.593073 28653 leader_election.cc:288] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [CANDIDATE]: Term 1 election: Requested vote from peers 0bacd61cc03d47989c9c03dfb3e0e014 (127.8.216.129:34843), a7a1b01260144d4abaa3e88f2c902f5e (127.8.216.131:40747)
I20260807 19:36:03.593642 28293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab6499e741443ca94c406993ba8ddc7" candidate_uuid: "0020cb5beb694100afe3acf0f2a14aa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bacd61cc03d47989c9c03dfb3e0e014"
I20260807 19:36:03.593776 28575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ab6499e741443ca94c406993ba8ddc7" candidate_uuid: "0020cb5beb694100afe3acf0f2a14aa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7a1b01260144d4abaa3e88f2c902f5e"
I20260807 19:36:03.593896 28293 raft_consensus.cc:3037] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:03.594064 28575 raft_consensus.cc:3037] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:03.596411 28293 raft_consensus.cc:2454] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0020cb5beb694100afe3acf0f2a14aa8 in term 1.
I20260807 19:36:03.596818 28575 raft_consensus.cc:2454] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0020cb5beb694100afe3acf0f2a14aa8 in term 1.
I20260807 19:36:03.596902 28362 leader_election.cc:302] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0020cb5beb694100afe3acf0f2a14aa8, 0bacd61cc03d47989c9c03dfb3e0e014; no voters:
I20260807 19:36:03.597270 28653 raft_consensus.cc:2781] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:03.597803 28653 raft_consensus.cc:686] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 1 LEADER]: Becoming Leader. State: Replica: 0020cb5beb694100afe3acf0f2a14aa8, State: Running, Role: LEADER
I20260807 19:36:03.598343 28653 consensus_queue.cc:237] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } }
I20260807 19:36:03.603199 28127 catalog_manager.cc:5495] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0020cb5beb694100afe3acf0f2a14aa8 (127.8.216.130). New cstate: current_term: 1 leader_uuid: "0020cb5beb694100afe3acf0f2a14aa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0020cb5beb694100afe3acf0f2a14aa8" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38279 } health_report { overall_health: HEALTHY } } }
I20260807 19:36:03.674345 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.666819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.8.216.130:38279)
0807 19:36:03.667043 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.667049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.667072 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:36:03.667313 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.667322 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.667468 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.667646 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:03.667978 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.669141 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.671369 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.672147 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.672275 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.673028 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:03.673733 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:03.673878 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.673958 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.674046 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:36:03.682837 28575 raft_consensus.cc:1261] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 1 FOLLOWER]: Refusing update from remote peer 0020cb5beb694100afe3acf0f2a14aa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:03.683043 28293 raft_consensus.cc:1261] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 1 FOLLOWER]: Refusing update from remote peer 0020cb5beb694100afe3acf0f2a14aa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:03.684059 28657 consensus_queue.cc:1035] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7a1b01260144d4abaa3e88f2c902f5e" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:03.684471 28653 consensus_queue.cc:1035] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bacd61cc03d47989c9c03dfb3e0e014" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:03.710446 28127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:03.710757 28127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:03.715127 28127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ece79d5312d4c4e9ba2f178d18ad585: Sending DeleteTablet for 3 replicas of tablet 7ab6499e741443ca94c406993ba8ddc7
I20260807 19:36:03.716436 28270 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:03 UTC) from {username='slave'} at 127.0.0.1:42316
I20260807 19:36:03.716595 28554 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:03 UTC) from {username='slave'} at 127.0.0.1:60016
I20260807 19:36:03.716980 28414 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ab6499e741443ca94c406993ba8ddc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:03 UTC) from {username='slave'} at 127.0.0.1:35312
I20260807 19:36:03.718694 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28197
I20260807 19:36:03.738277 28672 tablet_replica.cc:330] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: stopping tablet replica
I20260807 19:36:03.739022 28672 raft_consensus.cc:2229] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:03.739955 28672 raft_consensus.cc:2258] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:03.740859 28670 tablet_replica.cc:330] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: stopping tablet replica
I20260807 19:36:03.741429 28670 raft_consensus.cc:2229] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:03.742093 28670 raft_consensus.cc:2258] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:03.748471 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.720004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.8.216.129:34843)
0807 19:36:03.720239 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.720249 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.731189 (+ 10940us) server_negotiation.cc:407] Connection header received
0807 19:36:03.731825 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.731836 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.732020 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.732139 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:03.733053 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.734670 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.742603 (+ 7933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.743555 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.743694 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.746195 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:03.746211 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:03.746235 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:03.747235 (+ 1000us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:03.747312 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:03.747345 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:03.747464 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.747763 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.747944 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:36:03.750519 28336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:03.753245 28672 ts_tablet_manager.cc:1905] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:03.760637 28671 tablet_replica.cc:330] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: stopping tablet replica
I20260807 19:36:03.761112 28671 raft_consensus.cc:2229] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:03.761647 28671 raft_consensus.cc:2258] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:03.764227 28671 ts_tablet_manager.cc:1905] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:03.764989 28672 ts_tablet_manager.cc:1918] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:03.765193 28672 log.cc:1196] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/7ab6499e741443ca94c406993ba8ddc7
I20260807 19:36:03.765813 28672 ts_tablet_manager.cc:1939] T 7ab6499e741443ca94c406993ba8ddc7 P 0020cb5beb694100afe3acf0f2a14aa8: Deleting consensus metadata
I20260807 19:36:03.767812 28114 catalog_manager.cc:4841] TS 0020cb5beb694100afe3acf0f2a14aa8 (127.8.216.130:38279): tablet 7ab6499e741443ca94c406993ba8ddc7 (table test-table [id=652e6160ee3d401ca076d561f3ca0615]) successfully deleted
I20260807 19:36:03.770714 28671 ts_tablet_manager.cc:1918] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:03.770916 28671 log.cc:1196] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/7ab6499e741443ca94c406993ba8ddc7
I20260807 19:36:03.771492 28671 ts_tablet_manager.cc:1939] T 7ab6499e741443ca94c406993ba8ddc7 P a7a1b01260144d4abaa3e88f2c902f5e: Deleting consensus metadata
I20260807 19:36:03.773622 28113 catalog_manager.cc:4841] TS a7a1b01260144d4abaa3e88f2c902f5e (127.8.216.131:40747): tablet 7ab6499e741443ca94c406993ba8ddc7 (table test-table [id=652e6160ee3d401ca076d561f3ca0615]) successfully deleted
I20260807 19:36:03.751466 28670 ts_tablet_manager.cc:1905] T 7ab6499e741443ca94c406993ba8ddc7 P 0bacd61cc03d47989c9c03dfb3e0e014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:03.877254 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28197
W20260807 19:36:03.905311 28112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:34843 (error 108)
I20260807 19:36:03.906147 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28346
I20260807 19:36:03.907979 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.906705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50598)
0807 19:36:03.907632 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.907777 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":664,"threads_started":1}
W20260807 19:36:03.908304 28112 catalog_manager.cc:4584] TS 0bacd61cc03d47989c9c03dfb3e0e014 (127.8.216.129:34843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ab6499e741443ca94c406993ba8ddc7: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
I20260807 19:36:03.915493 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.907038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.8.216.130:38279)
0807 19:36:03.907223 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:03.907229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:03.907251 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:36:03.907402 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:03.907410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:03.907542 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:03.907658 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:03.908211 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.909446 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.911461 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:03.912005 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:03.912116 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:03.913637 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:03.913646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:03.913662 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:03.914522 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:03.914586 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:03.914628 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:03.914739 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:03.914942 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 19:36:03.915077 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:36:03.916460 28478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:03.962116 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.961618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50608)
0807 19:36:03.961854 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.961965 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:36:03.963189 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:03.962791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50622)
0807 19:36:03.962965 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:03.963033 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:36:04.023931 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.023347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50624)
0807 19:36:04.023671 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:04.023769 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260807 19:36:04.025148 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.024714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50636)
0807 19:36:04.024923 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:04.025002 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 19:36:04.035204 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28346
I20260807 19:36:04.059543 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28486
I20260807 19:36:04.066771 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.060443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37744 (local address 127.8.216.131:40747)
0807 19:36:04.060642 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:04.060650 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:04.060672 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:36:04.060906 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:04.060911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:04.060998 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:04.061113 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:04.061545 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:04.062279 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:04.063634 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:04.064035 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:04.064203 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:04.065357 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:04.065367 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:04.065383 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:04.065980 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:04.066030 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:04.066050 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:04.066180 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:04.066316 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 19:36:04.066433 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:36:04.067508 28619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:04.105785 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.105256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50644)
0807 19:36:04.105514 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:04.105634 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 19:36:04.106963 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.106539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50654)
0807 19:36:04.106752 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:04.106824 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 19:36:04.161370 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28486
I20260807 19:36:04.183892 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28096
I20260807 19:36:04.185187 28163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:04.300204 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:04.299678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34843 (local address 127.0.0.1:50662)
0807 19:36:04.299865 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:04.299968 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 19:36:04.300170 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28096
2026-08-07T19:36:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3209 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:04 dist-test-slave-8w9v krb5kdc[28683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:04 dist-test-slave-8w9v krb5kdc[28683](info): set up 2 sockets
Aug 07 19:36:04 dist-test-slave-8w9v krb5kdc[28683](info): commencing operation
krb5kdc: starting...
W20260807 19:36:05.478638 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:05 dist-test-slave-8w9v krb5kdc[28683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:05.815011 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41629
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:41629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:06.184571 28697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:06.184983 28697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:06.185199 28697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:06.193075 28697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:06.193279 28697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:06.193405 28697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:06.193528 28697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:06.193630 28697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:06.204499 28697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:41629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:41629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:06.206056 28697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:06.207535 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:06.220671 28702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:06.220742 28703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:06.220705 28705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:06.221963 28697 server_base.cc:1034] running on GCE node
I20260807 19:36:06.222777 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:06.224279 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:06.225507 28697 hybrid_clock.cc:648] HybridClock initialized: now 1786131366225453 us; error 68 us; skew 500 ppm
I20260807 19:36:06.225857 28697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:06 dist-test-slave-8w9v krb5kdc[28683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:06.229126 28697 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:06.230768 28697 webserver.cc:460] Webserver started at http://127.8.216.190:43117/ using document root <none> and password file <none>
I20260807 19:36:06.231487 28697 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:06.231637 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:06.231964 28697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:06.235100 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "0f2917c019e144d694e6b17f7d12056a"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "5a8b13b05714b6aa5519caf056d75d5b"
server_key_iv: "81931c0ddc925a988f490346a505f27c"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.235896 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "0f2917c019e144d694e6b17f7d12056a"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "5a8b13b05714b6aa5519caf056d75d5b"
server_key_iv: "81931c0ddc925a988f490346a505f27c"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.241359 28697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:36:06.245276 28713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:06.246716 28697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:36:06.246927 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "0f2917c019e144d694e6b17f7d12056a"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "5a8b13b05714b6aa5519caf056d75d5b"
server_key_iv: "81931c0ddc925a988f490346a505f27c"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.247145 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:06.288932 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:06.293717 28697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:06.294034 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:06.314198 28697 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:41629
I20260807 19:36:06.314232 28771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:41629 every 8 connection(s)
I20260807 19:36:06.315649 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:06.318430 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28697
I20260807 19:36:06.318652 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:06.319234 9058 external_mini_cluster.cc:1428] Setting key 70a1399a7d3e9c807f33e0da7cfd7771
I20260807 19:36:06.320240 28772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:06.332832 28772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: Bootstrap starting.
Aug 07 19:36:06 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:06.337042 28772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:06.338183 28772 log.cc:824] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:06.341739 28772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: No bootstrap required, opened a new log
I20260807 19:36:06.342833 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:06.321327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.8.216.190:41629)
0807 19:36:06.322141 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:06.322154 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:06.322194 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:36:06.323831 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:06.323852 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:06.324320 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:06.324742 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:06.326412 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:06.327759 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:06.329267 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:06.330009 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:06.330207 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:06.334996 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:06.335004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:06.335018 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:06.335066 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:06.338380 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:06.339295 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:06.339305 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:06.339316 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:06.339532 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:06.339972 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:06.339975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:06.339978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:06.340561 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:06.340782 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:06.341316 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 19:36:06.341734 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":369,"threads_started":1}
I20260807 19:36:06.349787 28772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } }
I20260807 19:36:06.350224 28772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:06.350340 28772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f2917c019e144d694e6b17f7d12056a, State: Initialized, Role: FOLLOWER
I20260807 19:36:06.350942 28772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } }
I20260807 19:36:06.351202 28772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:06.351356 28772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:06.351554 28772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:06.353952 28772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } }
I20260807 19:36:06.354485 28772 leader_election.cc:302] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f2917c019e144d694e6b17f7d12056a; no voters:
I20260807 19:36:06.355208 28772 leader_election.cc:288] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:06.355331 28778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:06.356074 28778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [term 1 LEADER]: Becoming Leader. State: Replica: 0f2917c019e144d694e6b17f7d12056a, State: Running, Role: LEADER
I20260807 19:36:06.356688 28778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } }
I20260807 19:36:06.357699 28772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:06.359591 28779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f2917c019e144d694e6b17f7d12056a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } } }
I20260807 19:36:06.360144 28779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:06.361344 28780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f2917c019e144d694e6b17f7d12056a. Latest consensus state: current_term: 1 leader_uuid: "0f2917c019e144d694e6b17f7d12056a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f2917c019e144d694e6b17f7d12056a" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 41629 } } }
I20260807 19:36:06.362632 28780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:06.362862 28783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:06.368956 28783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:06.378832 28783 catalog_manager.cc:1347] Generated new cluster ID: 6603323a45cd4696ab0da9097b4ce7fc
I20260807 19:36:06.378976 28783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:06.407881 28783 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:06.409334 28783 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:06.416551 28783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: Generated new TSK 0
I20260807 19:36:06.417388 28783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:36:06.489701 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:06.850670 28799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:06.851048 28799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:06.851205 28799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:06.858716 28799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:06.858883 28799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:06.859067 28799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:36:06.869477 28799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:06.870941 28799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:06.872296 28799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:06.886116 28807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:06.886170 28805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:06.886462 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:06.886894 28799 server_base.cc:1034] running on GCE node
I20260807 19:36:06.887593 28799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:06.888682 28799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:06.889914 28799 hybrid_clock.cc:648] HybridClock initialized: now 1786131366889849 us; error 38 us; skew 500 ppm
I20260807 19:36:06.890264 28799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:06 dist-test-slave-8w9v krb5kdc[28683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:06.893661 28799 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:36:06.895210 28799 webserver.cc:460] Webserver started at http://127.8.216.129:43235/ using document root <none> and password file <none>
I20260807 19:36:06.895988 28799 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:06.896201 28799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:06.896553 28799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:06.899643 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "9cdafafbad8b4b94b3e34870f07fd7b2"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "172d5bf1dde8320fd8e5378207496433"
server_key_iv: "a3d13fa86a06381ba403a9dfba6e0cb5"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.900555 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "9cdafafbad8b4b94b3e34870f07fd7b2"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "172d5bf1dde8320fd8e5378207496433"
server_key_iv: "a3d13fa86a06381ba403a9dfba6e0cb5"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.906136 28799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 19:36:06.910153 28814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:06.911734 28799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:36:06.911957 28799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "9cdafafbad8b4b94b3e34870f07fd7b2"
format_stamp: "Formatted at 2026-08-07 19:36:06 on dist-test-slave-8w9v"
server_key: "172d5bf1dde8320fd8e5378207496433"
server_key_iv: "a3d13fa86a06381ba403a9dfba6e0cb5"
server_key_version: "encryptionkey@0"
I20260807 19:36:06.912232 28799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:06.946895 28799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:06.951412 28799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:06.951767 28799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:06.952913 28799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:06.954368 28799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:06.954490 28799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:06.954654 28799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:06.954731 28799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:06.990151 28799 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:39159
I20260807 19:36:06.990203 28941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:39159 every 8 connection(s)
I20260807 19:36:06.991549 28799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:36:06.992775 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28799
I20260807 19:36:06.992988 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:36:06.993587 9058 external_mini_cluster.cc:1428] Setting key 3d0771dbf7c21825f2cf1da82d634e19
Aug 07 19:36:07 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:07.017462 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:06.994568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:58193 (local address 127.8.216.190:41629)
0807 19:36:06.995057 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:06.995063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:06.996059 (+ 996us) server_negotiation.cc:407] Connection header received
0807 19:36:06.999004 (+ 2945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:06.999009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:06.999078 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:06.999175 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:07.001442 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.002148 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.004805 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.005213 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.005318 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:07.010478 (+ 5160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:07.010488 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:07.010496 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:07.010555 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:07.013717 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:07.014577 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:07.014582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:07.014584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:07.014722 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:07.015214 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:07.015219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:07.015221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:07.015539 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:07.015738 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:07.016810 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0807 19:36:07.017066 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":255,"threads_started":1}
I20260807 19:36:07.018953 28945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:06.995073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41629 (local address 127.8.216.129:58193)
0807 19:36:06.995857 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:06.995894 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:06.998679 (+ 2785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:06.999481 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:06.999495 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:07.000334 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:07.001212 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:07.001241 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.002316 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.002324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:07.004627 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:07.004648 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.005481 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.005491 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:07.005850 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:07.006704 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:07.006748 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:07.010051 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:07.013900 (+ 3849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:07.013913 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:07.013940 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:07.014407 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:07.014843 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:07.014849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:07.014854 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:07.015068 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:07.015699 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:07.015711 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:07.016321 (+ 610us) client_negotiation.cc:769] Sending connection context
0807 19:36:07.016684 (+ 363us) client_negotiation.cc:240] Negotiation successful
0807 19:36:07.017336 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":353,"threads_started":1}
I20260807 19:36:07.020737 28942 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41629
I20260807 19:36:07.021072 28942 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:07.021826 28942 heartbeater.cc:511] Master 127.8.216.190:41629 requested a full tablet report, sending...
I20260807 19:36:07.024307 28732 ts_manager.cc:194] Registered new tserver with Master: 9cdafafbad8b4b94b3e34870f07fd7b2 (127.8.216.129:39159)
I20260807 19:36:07.026563 28732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:58193
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:36:07.071403 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:07.439843 28948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:07.440222 28948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:07.440375 28948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:07.447745 28948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:07.447856 28948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:07.448040 28948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:36:07.460361 28948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:07.462139 28948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:07.463972 28948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:07.477905 28953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:07.478005 28954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:07.477967 28956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:07.478837 28948 server_base.cc:1034] running on GCE node
I20260807 19:36:07.479553 28948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:07.480624 28948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:07.481911 28948 hybrid_clock.cc:648] HybridClock initialized: now 1786131367481851 us; error 41 us; skew 500 ppm
I20260807 19:36:07.482251 28948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:07 dist-test-slave-8w9v krb5kdc[28683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:07.485532 28948 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:36:07.487017 28948 webserver.cc:460] Webserver started at http://127.8.216.130:42207/ using document root <none> and password file <none>
I20260807 19:36:07.487671 28948 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:07.487772 28948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:07.488056 28948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:07.491174 28948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "01d2995fcdf1435ca2302536016c06a9"
format_stamp: "Formatted at 2026-08-07 19:36:07 on dist-test-slave-8w9v"
server_key: "cbe8029545995512e0826d9bda2348b5"
server_key_iv: "b61187f7da2fca53b7eb78e4e10f444e"
server_key_version: "encryptionkey@0"
I20260807 19:36:07.491952 28948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "01d2995fcdf1435ca2302536016c06a9"
format_stamp: "Formatted at 2026-08-07 19:36:07 on dist-test-slave-8w9v"
server_key: "cbe8029545995512e0826d9bda2348b5"
server_key_iv: "b61187f7da2fca53b7eb78e4e10f444e"
server_key_version: "encryptionkey@0"
I20260807 19:36:07.497464 28948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:36:07.501077 28963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:07.502445 28948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:07.502588 28948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "01d2995fcdf1435ca2302536016c06a9"
format_stamp: "Formatted at 2026-08-07 19:36:07 on dist-test-slave-8w9v"
server_key: "cbe8029545995512e0826d9bda2348b5"
server_key_iv: "b61187f7da2fca53b7eb78e4e10f444e"
server_key_version: "encryptionkey@0"
I20260807 19:36:07.502781 28948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:07.520478 28948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:07.524963 28948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:07.525228 28948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:07.526190 28948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:07.527570 28948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:07.527648 28948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:07.527741 28948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:07.527787 28948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:07.562587 28948 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:42265
I20260807 19:36:07.562623 29087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:42265 every 8 connection(s)
I20260807 19:36:07.564100 28948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:36:07.564642 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 28948
I20260807 19:36:07.564852 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:36:07.565431 9058 external_mini_cluster.cc:1428] Setting key e1c228bf6fb37f38caa847b1f009629f
Aug 07 19:36:07 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:07.587622 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:07.567090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:54017 (local address 127.8.216.190:41629)
0807 19:36:07.567617 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:07.567623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:07.568837 (+ 1214us) server_negotiation.cc:407] Connection header received
0807 19:36:07.570932 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:07.570938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:07.571039 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:07.571167 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:07.572878 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.573625 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.575226 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.575699 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.575846 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:07.579918 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:07.579923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:07.579926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:07.579961 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:07.583683 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:07.584724 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:07.584733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:07.584739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:07.584917 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:07.585406 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:07.585414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:07.585420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:07.585841 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:07.586022 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:07.586807 (+ 785us) server_negotiation.cc:299] Negotiation successful
0807 19:36:07.587083 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260807 19:36:07.588418 29092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:07.567766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41629 (local address 127.8.216.130:54017)
0807 19:36:07.568665 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:07.568713 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:07.570609 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:07.571439 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:07.571449 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:07.572014 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:07.572693 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:07.572712 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.573808 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.573812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:07.575072 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:07.575085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:07.575978 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:07.575987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:07.576293 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:07.577032 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:07.577060 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:07.579653 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:07.583878 (+ 4225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:07.583886 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:07.583902 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:07.584554 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:07.585035 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:07.585038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:07.585040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:07.585271 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:07.585976 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:07.585982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:07.586400 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 19:36:07.586642 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 19:36:07.587074 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":438,"threads_started":1}
I20260807 19:36:07.590307 29088 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41629
I20260807 19:36:07.590667 29088 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:07.591495 29088 heartbeater.cc:511] Master 127.8.216.190:41629 requested a full tablet report, sending...
I20260807 19:36:07.593151 28732 ts_manager.cc:194] Registered new tserver with Master: 01d2995fcdf1435ca2302536016c06a9 (127.8.216.130:42265)
I20260807 19:36:07.594035 28732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:54017
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:36:07.637508 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:07.994546 29096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:07.994827 29096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:07.994985 29096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:08.002530 29096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:08.002645 29096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:08.002815 29096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:36:08.013304 29096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:08.014652 29096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:08.016053 29096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:08.029559 29103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:08.029575 29106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:08.029834 28942 heartbeater.cc:503] Master 127.8.216.190:41629 was elected leader, sending a full tablet report...
W20260807 19:36:08.030187 29102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:08.030678 29096 server_base.cc:1034] running on GCE node
I20260807 19:36:08.031347 29096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:08.032528 29096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:08.033857 29096 hybrid_clock.cc:648] HybridClock initialized: now 1786131368033811 us; error 52 us; skew 500 ppm
I20260807 19:36:08.034231 29096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:08 dist-test-slave-8w9v krb5kdc[28683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:08.037654 29096 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:36:08.039280 29096 webserver.cc:460] Webserver started at http://127.8.216.131:43713/ using document root <none> and password file <none>
I20260807 19:36:08.039988 29096 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:08.040098 29096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:08.040462 29096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:08.043506 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "26dd1016a1554082ba350ab5af6766e5"
format_stamp: "Formatted at 2026-08-07 19:36:08 on dist-test-slave-8w9v"
server_key: "bf6cc1a79a68be9876bf22f314f31454"
server_key_iv: "973256a8c754cfe44b69d97ab5f7061c"
server_key_version: "encryptionkey@0"
I20260807 19:36:08.044368 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "26dd1016a1554082ba350ab5af6766e5"
format_stamp: "Formatted at 2026-08-07 19:36:08 on dist-test-slave-8w9v"
server_key: "bf6cc1a79a68be9876bf22f314f31454"
server_key_iv: "973256a8c754cfe44b69d97ab5f7061c"
server_key_version: "encryptionkey@0"
I20260807 19:36:08.049759 29096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:36:08.053457 29113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:08.054832 29096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:08.055011 29096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "26dd1016a1554082ba350ab5af6766e5"
format_stamp: "Formatted at 2026-08-07 19:36:08 on dist-test-slave-8w9v"
server_key: "bf6cc1a79a68be9876bf22f314f31454"
server_key_iv: "973256a8c754cfe44b69d97ab5f7061c"
server_key_version: "encryptionkey@0"
I20260807 19:36:08.055238 29096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:08.084470 29096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:08.088769 29096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:08.089087 29096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:08.090078 29096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:08.091512 29096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:08.091621 29096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:08.091777 29096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:08.091851 29096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:08.127089 29096 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:39141
I20260807 19:36:08.127147 29236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:39141 every 8 connection(s)
I20260807 19:36:08.128525 29096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:36:08.129197 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29096
I20260807 19:36:08.129405 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:36:08.130040 9058 external_mini_cluster.cc:1428] Setting key 9546eb8db04294b25c9508d93ed93e7e
Aug 07 19:36:08 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:08.149960 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.131310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:58485 (local address 127.8.216.190:41629)
0807 19:36:08.131839 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.131843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.132907 (+ 1064us) server_negotiation.cc:407] Connection header received
0807 19:36:08.135038 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.135044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.135146 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.135276 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:08.136868 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.137640 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.139185 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.139615 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.139753 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.143412 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:08.143430 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:08.143433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:08.143467 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:08.146724 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.147492 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.147498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.147503 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.147628 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.148087 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.148092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.148096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.148434 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:08.148608 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.149312 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.149559 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":266,"threads_started":1}
I20260807 19:36:08.150982 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.131877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:41629 (local address 127.8.216.131:58485)
0807 19:36:08.132760 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.132806 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.134749 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.135538 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.135546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.136072 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:08.136702 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.136719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.137788 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.137793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.139037 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.139054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.139847 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.139854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.140101 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.140877 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:08.140898 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:08.143201 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:08.146865 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:08.146876 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:08.146893 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:08.147353 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:08.147742 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:08.147746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:08.147750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:08.147962 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:08.148581 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:08.148590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:08.148944 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.149215 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.149777 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":361,"threads_started":1}
I20260807 19:36:08.152488 29237 heartbeater.cc:347] Connected to a master server at 127.8.216.190:41629
I20260807 19:36:08.152753 29237 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:08.153420 29237 heartbeater.cc:511] Master 127.8.216.190:41629 requested a full tablet report, sending...
I20260807 19:36:08.154821 28732 ts_manager.cc:194] Registered new tserver with Master: 26dd1016a1554082ba350ab5af6766e5 (127.8.216.131:39141)
I20260807 19:36:08.155696 28732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:58485
I20260807 19:36:08.158913 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:36:08.176729 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.163264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.8.216.190:41629)
0807 19:36:08.163443 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.163448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.164009 (+ 561us) server_negotiation.cc:407] Connection header received
0807 19:36:08.164242 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.164246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.164377 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.164464 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:08.165974 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.166687 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.167967 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.168440 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.168543 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.171342 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:08.171346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:08.171350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:08.171385 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:08.174115 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.174869 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.174873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.174875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.175005 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.175409 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.175412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.175414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.175722 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:08.175932 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.176335 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.176494 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:36:08.180999 28732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:08.188021 28732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:08.216305 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.201210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.8.216.131:39141)
0807 19:36:08.206590 (+ 5380us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.206599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.206638 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:36:08.206738 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.206745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.206983 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.208000 (+ 1017us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:08.208353 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.210080 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.213314 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.214880 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.215019 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.215274 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.215726 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.215908 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5122,"thread_start_us":4920,"threads_started":1}
I20260807 19:36:08.216544 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.200008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:42265 (local address 127.0.0.1:46704)
0807 19:36:08.201985 (+ 1977us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.202032 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.202194 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.206096 (+ 3902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.206104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.206141 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:08.206617 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.206633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.210323 (+ 3690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.210334 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.213698 (+ 3364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.213713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.215461 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.215467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.215644 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.215674 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.215752 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.215864 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1737,"mutex_wait_us":80,"thread_start_us":5468,"threads_started":1}
I20260807 19:36:08.216723 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.200919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:39141 (local address 127.0.0.1:37636)
0807 19:36:08.205739 (+ 4820us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.205759 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.205878 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.207207 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.207215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.207244 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:08.207722 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.207737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.210263 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.210272 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.213107 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.213125 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.215102 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.215110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.215343 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.215412 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.215557 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.215771 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4700,"mutex_wait_us":121,"thread_start_us":677,"threads_started":1}
I20260807 19:36:08.217533 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.200267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.8.216.130:42265)
0807 19:36:08.205355 (+ 5088us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.205366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.205401 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:36:08.205521 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.205530 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.205797 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.206926 (+ 1129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:08.207185 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.210185 (+ 3000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.213876 (+ 3691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.215235 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.215401 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.215598 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.217006 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.217208 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4925,"thread_start_us":4793,"threads_started":1}
I20260807 19:36:08.220983 29016 tablet_service.cc:1467] Processing CreateTablet for tablet 0950ef0479024fb7a56abf0f8af7667b (DEFAULT_TABLE table=test-table [id=48a77a9b0b9a402ea08c2522b60095ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:08.220829 29162 tablet_service.cc:1467] Processing CreateTablet for tablet 0950ef0479024fb7a56abf0f8af7667b (DEFAULT_TABLE table=test-table [id=48a77a9b0b9a402ea08c2522b60095ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:08.222385 29016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0950ef0479024fb7a56abf0f8af7667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:08.222394 29162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0950ef0479024fb7a56abf0f8af7667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:08.222810 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.206576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:39159 (local address 127.0.0.1:54254)
0807 19:36:08.213866 (+ 7290us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.213886 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.214023 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.214715 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.214721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.214748 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:08.215202 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.215217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.217476 (+ 2259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.217481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.219964 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.219980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.221908 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.221918 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.222088 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.222123 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.222227 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.222368 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7115,"thread_start_us":6933,"threads_started":1}
I20260807 19:36:08.222834 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.206245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.8.216.129:39159)
0807 19:36:08.212009 (+ 5764us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.212020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.214009 (+ 1989us) server_negotiation.cc:407] Connection header received
0807 19:36:08.214194 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.214202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.214490 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.214698 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:08.215444 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.217299 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.220185 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.221668 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.221822 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.222076 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.222297 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.222495 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5339,"thread_start_us":5183,"threads_started":1}
I20260807 19:36:08.226809 28865 tablet_service.cc:1467] Processing CreateTablet for tablet 0950ef0479024fb7a56abf0f8af7667b (DEFAULT_TABLE table=test-table [id=48a77a9b0b9a402ea08c2522b60095ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:08.228338 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0950ef0479024fb7a56abf0f8af7667b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:08.235324 29256 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Bootstrap starting.
I20260807 19:36:08.238340 29257 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Bootstrap starting.
I20260807 19:36:08.239193 29256 tablet_bootstrap.cc:654] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:08.241521 29256 log.cc:824] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:08.242146 29257 tablet_bootstrap.cc:654] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:08.243338 29257 log.cc:824] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:08.244158 29259 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Bootstrap starting.
I20260807 19:36:08.245487 29256 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: No bootstrap required, opened a new log
I20260807 19:36:08.245888 29256 ts_tablet_manager.cc:1397] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:36:08.247447 29259 tablet_bootstrap.cc:654] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:08.248821 29259 log.cc:824] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:08.251223 29257 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: No bootstrap required, opened a new log
I20260807 19:36:08.251598 29257 ts_tablet_manager.cc:1397] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260807 19:36:08.252028 29259 tablet_bootstrap.cc:492] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: No bootstrap required, opened a new log
I20260807 19:36:08.252388 29259 ts_tablet_manager.cc:1397] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260807 19:36:08.257412 29256 raft_consensus.cc:348] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.258000 29256 raft_consensus.cc:374] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:08.258116 29256 raft_consensus.cc:729] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01d2995fcdf1435ca2302536016c06a9, State: Initialized, Role: FOLLOWER
I20260807 19:36:08.258821 29256 consensus_queue.cc:260] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.260681 29259 raft_consensus.cc:348] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.261116 29259 raft_consensus.cc:374] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:08.261238 29259 raft_consensus.cc:729] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cdafafbad8b4b94b3e34870f07fd7b2, State: Initialized, Role: FOLLOWER
I20260807 19:36:08.261948 29259 consensus_queue.cc:260] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.263949 29259 ts_tablet_manager.cc:1428] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260807 19:36:08.264292 29257 raft_consensus.cc:348] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.264870 29257 raft_consensus.cc:374] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:08.264979 29257 raft_consensus.cc:729] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26dd1016a1554082ba350ab5af6766e5, State: Initialized, Role: FOLLOWER
I20260807 19:36:08.265794 29257 consensus_queue.cc:260] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.267387 29256 ts_tablet_manager.cc:1428] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Time spent starting tablet: real 0.021s user 0.014s sys 0.009s
I20260807 19:36:08.268050 29088 heartbeater.cc:503] Master 127.8.216.190:41629 was elected leader, sending a full tablet report...
I20260807 19:36:08.276969 29237 heartbeater.cc:503] Master 127.8.216.190:41629 was elected leader, sending a full tablet report...
I20260807 19:36:08.278064 29257 ts_tablet_manager.cc:1428] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Time spent starting tablet: real 0.026s user 0.013s sys 0.007s
W20260807 19:36:08.317755 29089 tablet.cc:2369] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:08.380654 29238 tablet.cc:2369] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:08.494781 28943 tablet.cc:2369] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:08.526481 29264 raft_consensus.cc:482] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:08.526788 29264 raft_consensus.cc:504] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.528498 29264 leader_election.cc:288] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01d2995fcdf1435ca2302536016c06a9 (127.8.216.130:42265), 9cdafafbad8b4b94b3e34870f07fd7b2 (127.8.216.129:39159)
I20260807 19:36:08.535247 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.528986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:49661 (local address 127.8.216.130:42265)
0807 19:36:08.529218 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.529224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.529250 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:36:08.529397 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.529405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.529498 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.529612 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:08.530254 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.531339 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.533216 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.534336 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.534460 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.534515 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.534876 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.534986 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:36:08.535331 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.528833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:42265 (local address 127.8.216.131:49661)
0807 19:36:08.529068 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.529089 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.529228 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.529810 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.529814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.529847 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:08.530039 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.530071 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.531481 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.531487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.533040 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.533054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.534495 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.534502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.534704 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.534740 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.534847 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.535005 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:36:08.536024 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.529629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:42891 (local address 127.8.216.129:39159)
0807 19:36:08.529840 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.529846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.530104 (+ 258us) server_negotiation.cc:407] Connection header received
0807 19:36:08.530239 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.530243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.530304 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.530374 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:08.530998 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.531921 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.533985 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.535043 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.535200 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.535269 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.535655 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.535790 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:36:08.536080 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.529290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:39159 (local address 127.8.216.131:42891)
0807 19:36:08.529991 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:08.530006 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:08.530103 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:08.530430 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:08.530434 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:08.530457 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:08.530843 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.530855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.532080 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.532084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.533807 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:08.533823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.535269 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.535278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:08.535493 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.535531 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:36:08.535626 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:36:08.535762 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":373,"threads_started":1}
I20260807 19:36:08.536618 29040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0950ef0479024fb7a56abf0f8af7667b" candidate_uuid: "26dd1016a1554082ba350ab5af6766e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01d2995fcdf1435ca2302536016c06a9" is_pre_election: true
I20260807 19:36:08.537231 29040 raft_consensus.cc:2454] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26dd1016a1554082ba350ab5af6766e5 in term 0.
I20260807 19:36:08.537247 28890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0950ef0479024fb7a56abf0f8af7667b" candidate_uuid: "26dd1016a1554082ba350ab5af6766e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" is_pre_election: true
I20260807 19:36:08.537807 28890 raft_consensus.cc:2454] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26dd1016a1554082ba350ab5af6766e5 in term 0.
I20260807 19:36:08.538029 29114 leader_election.cc:302] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01d2995fcdf1435ca2302536016c06a9, 26dd1016a1554082ba350ab5af6766e5; no voters:
I20260807 19:36:08.538466 29264 raft_consensus.cc:2781] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:08.538625 29264 raft_consensus.cc:482] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:08.538739 29264 raft_consensus.cc:3037] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:08.541312 29264 raft_consensus.cc:504] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.541985 29264 leader_election.cc:288] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [CANDIDATE]: Term 1 election: Requested vote from peers 01d2995fcdf1435ca2302536016c06a9 (127.8.216.130:42265), 9cdafafbad8b4b94b3e34870f07fd7b2 (127.8.216.129:39159)
I20260807 19:36:08.542434 29040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0950ef0479024fb7a56abf0f8af7667b" candidate_uuid: "26dd1016a1554082ba350ab5af6766e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01d2995fcdf1435ca2302536016c06a9"
I20260807 19:36:08.542520 28890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0950ef0479024fb7a56abf0f8af7667b" candidate_uuid: "26dd1016a1554082ba350ab5af6766e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2"
I20260807 19:36:08.542659 29040 raft_consensus.cc:3037] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:08.542778 28890 raft_consensus.cc:3037] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:08.545138 29040 raft_consensus.cc:2454] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26dd1016a1554082ba350ab5af6766e5 in term 1.
I20260807 19:36:08.545214 28890 raft_consensus.cc:2454] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26dd1016a1554082ba350ab5af6766e5 in term 1.
I20260807 19:36:08.545651 29114 leader_election.cc:302] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01d2995fcdf1435ca2302536016c06a9, 26dd1016a1554082ba350ab5af6766e5; no voters:
I20260807 19:36:08.545989 29264 raft_consensus.cc:2781] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:08.546484 29264 raft_consensus.cc:686] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 1 LEADER]: Becoming Leader. State: Replica: 26dd1016a1554082ba350ab5af6766e5, State: Running, Role: LEADER
I20260807 19:36:08.547061 29264 consensus_queue.cc:237] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } }
I20260807 19:36:08.552430 28731 catalog_manager.cc:5495] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26dd1016a1554082ba350ab5af6766e5 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "26dd1016a1554082ba350ab5af6766e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26dd1016a1554082ba350ab5af6766e5" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 39141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 } health_report { overall_health: UNKNOWN } } }
I20260807 19:36:08.584066 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.578359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.8.216.131:39141)
0807 19:36:08.578560 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.578564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.578682 (+ 118us) server_negotiation.cc:407] Connection header received
0807 19:36:08.578949 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.578954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.579042 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.579183 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:08.579588 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.580431 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.581817 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.582278 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.582387 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.582947 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:08.583483 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:08.583616 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.583799 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.583845 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:36:08.592298 29040 raft_consensus.cc:1261] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 1 FOLLOWER]: Refusing update from remote peer 26dd1016a1554082ba350ab5af6766e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:08.592296 28890 raft_consensus.cc:1261] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 1 FOLLOWER]: Refusing update from remote peer 26dd1016a1554082ba350ab5af6766e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:08.593278 29264 consensus_queue.cc:1035] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cdafafbad8b4b94b3e34870f07fd7b2" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 39159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:08.593614 29266 consensus_queue.cc:1035] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01d2995fcdf1435ca2302536016c06a9" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 42265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:08.604969 29270 mvcc.cc:205] Tried to move back new op lower bound from 7315994085743529984 to 7315994085573398528. Current Snapshot: MvccSnapshot[applied={T|T < 7315994085743529984}]
I20260807 19:36:08.613428 29271 mvcc.cc:205] Tried to move back new op lower bound from 7315994085743529984 to 7315994085573398528. Current Snapshot: MvccSnapshot[applied={T|T < 7315994085743529984}]
I20260807 19:36:08.623782 28732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:08.624207 28732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:08.629051 28732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f2917c019e144d694e6b17f7d12056a: Sending DeleteTablet for 3 replicas of tablet 0950ef0479024fb7a56abf0f8af7667b
I20260807 19:36:08.630290 29162 tablet_service.cc:1514] Processing DeleteTablet for tablet 0950ef0479024fb7a56abf0f8af7667b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:08 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:37636
I20260807 19:36:08.630350 29016 tablet_service.cc:1514] Processing DeleteTablet for tablet 0950ef0479024fb7a56abf0f8af7667b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:08 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:46704
I20260807 19:36:08.630362 28865 tablet_service.cc:1514] Processing DeleteTablet for tablet 0950ef0479024fb7a56abf0f8af7667b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:08 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:54254
I20260807 19:36:08.631403 28864 tablet_service.cc:1514] Processing DeleteTablet for tablet 0950ef0479024fb7a56abf0f8af7667b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:08 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:54254
I20260807 19:36:08.633558 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28799
I20260807 19:36:08.642999 29280 tablet_replica.cc:330] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: stopping tablet replica
I20260807 19:36:08.643697 29282 tablet_replica.cc:330] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: stopping tablet replica
I20260807 19:36:08.643884 29281 tablet_replica.cc:330] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: stopping tablet replica
I20260807 19:36:08.644400 29281 raft_consensus.cc:2229] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:08.644860 29282 raft_consensus.cc:2229] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:08.644492 29280 raft_consensus.cc:2229] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:08.645153 29281 raft_consensus.cc:2258] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:08.646440 29282 raft_consensus.cc:2258] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:08.646755 29280 raft_consensus.cc:2258] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:08.647224 29281 ts_tablet_manager.cc:1905] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:08.650135 29282 ts_tablet_manager.cc:1905] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:08.651499 29280 ts_tablet_manager.cc:1905] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:08.655751 29281 ts_tablet_manager.cc:1918] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:08.656280 29281 log.cc:1196] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/0950ef0479024fb7a56abf0f8af7667b
I20260807 19:36:08.656896 29281 ts_tablet_manager.cc:1939] T 0950ef0479024fb7a56abf0f8af7667b P 26dd1016a1554082ba350ab5af6766e5: Deleting consensus metadata
I20260807 19:36:08.658933 28716 catalog_manager.cc:4841] TS 26dd1016a1554082ba350ab5af6766e5 (127.8.216.131:39141): tablet 0950ef0479024fb7a56abf0f8af7667b (table test-table [id=48a77a9b0b9a402ea08c2522b60095ff]) successfully deleted
I20260807 19:36:08.659018 29280 ts_tablet_manager.cc:1918] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:08.659180 29280 log.cc:1196] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/0950ef0479024fb7a56abf0f8af7667b
I20260807 19:36:08.659691 29280 ts_tablet_manager.cc:1939] T 0950ef0479024fb7a56abf0f8af7667b P 01d2995fcdf1435ca2302536016c06a9: Deleting consensus metadata
Aug 07 19:36:08 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:36:08.661568 28714 catalog_manager.cc:4841] TS 01d2995fcdf1435ca2302536016c06a9 (127.8.216.130:42265): tablet 0950ef0479024fb7a56abf0f8af7667b (table test-table [id=48a77a9b0b9a402ea08c2522b60095ff]) successfully deleted
I20260807 19:36:08.665460 29282 ts_tablet_manager.cc:1918] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:08.665647 29282 log.cc:1196] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/0950ef0479024fb7a56abf0f8af7667b
I20260807 19:36:08.666227 29282 ts_tablet_manager.cc:1939] T 0950ef0479024fb7a56abf0f8af7667b P 9cdafafbad8b4b94b3e34870f07fd7b2: Deleting consensus metadata
I20260807 19:36:08.668172 28714 catalog_manager.cc:4841] TS 9cdafafbad8b4b94b3e34870f07fd7b2 (127.8.216.129:39159): tablet 0950ef0479024fb7a56abf0f8af7667b (table test-table [id=48a77a9b0b9a402ea08c2522b60095ff]) successfully deleted
I20260807 19:36:08.668632 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.636336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.8.216.129:39159)
0807 19:36:08.636525 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.636531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.639413 (+ 2882us) server_negotiation.cc:407] Connection header received
0807 19:36:08.639646 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.639654 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.639739 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.639842 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:08.645207 (+ 5365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.647309 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.649451 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.650119 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.650224 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.661703 (+ 11479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:08.661723 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:08.661743 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:08.661811 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:08.665325 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.666240 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.666252 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.666261 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.666480 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.666904 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.666909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.666913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.667309 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:08.667646 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.667954 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.668210 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:36:08.669340 28936 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 19:36:08.669461 28714 catalog_manager.cc:4808] TS 9cdafafbad8b4b94b3e34870f07fd7b2 (127.8.216.129:39159): delete failed for tablet 0950ef0479024fb7a56abf0f8af7667b because the tablet was not found. No further retry: Not found: Tablet not found: 0950ef0479024fb7a56abf0f8af7667b
I20260807 19:36:08.765020 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28799
I20260807 19:36:08.795614 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28948
Aug 07 19:36:08 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:36:08.814169 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.796596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.8.216.130:42265)
0807 19:36:08.796782 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.796786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.796832 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:36:08.797062 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.797066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.797134 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.797232 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:08.798974 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.799699 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.801652 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.802067 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.802166 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.807753 (+ 5587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:08.807773 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:08.807791 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:08.807832 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:08.810895 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.811750 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.811757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.811760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.811902 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.812420 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.812424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.812426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.812838 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:08.813202 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.813524 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.813767 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:36:08.815057 29083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:08.908241 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28948
I20260807 19:36:08.936851 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29096
Aug 07 19:36:08 dist-test-slave-8w9v krb5kdc[28683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:36:08.955535 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:08.937728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37660 (local address 127.8.216.131:39141)
0807 19:36:08.937934 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:08.937940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:08.937966 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:36:08.938121 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:08.938127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:08.938211 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:08.938332 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:08.939656 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.940937 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.942192 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:08.942907 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:08.943075 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:08.947763 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:08.947778 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:08.947798 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:08.947850 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:08.952099 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.953179 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.953189 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.953193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.953395 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:08.953773 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:08.953778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:08.953781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:08.954299 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:08.954656 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:08.954859 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 19:36:08.955117 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:36:08.956445 29232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:09.083585 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29096
I20260807 19:36:09.108172 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 28697
I20260807 19:36:09.109335 28767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:09.224740 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 28697
2026-08-07T19:36:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4925 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T19:36:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:09Z Disabled control of system clock
I20260807 19:36:09.325657 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35967
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:35967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:09.680513 29295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:09.680817 29295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:09.680907 29295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:09.688501 29295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:09.688621 29295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:09.688659 29295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:09.688719 29295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:09.688784 29295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:09.699477 29295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:35967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:35967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:09.701056 29295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:09.702862 29295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:09.715009 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:09.715245 29300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:09.715279 29304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:09.716229 29295 server_base.cc:1034] running on GCE node
I20260807 19:36:09.717015 29295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:09.718387 29295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:09.719600 29295 hybrid_clock.cc:648] HybridClock initialized: now 1786131369719544 us; error 49 us; skew 500 ppm
I20260807 19:36:09.719960 29295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:09.722308 29295 webserver.cc:460] Webserver started at http://127.8.216.190:46321/ using document root <none> and password file <none>
I20260807 19:36:09.723033 29295 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:09.723155 29295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:09.723505 29295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:09.726512 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "d8d12730d746408296e1831de8e6607b"
format_stamp: "Formatted at 2026-08-07 19:36:09 on dist-test-slave-8w9v"
server_key: "376abeece54d8dcca0593dc67e58f53c"
server_key_iv: "01283b71e45ff372045055bcb0f0e437"
server_key_version: "encryptionkey@0"
I20260807 19:36:09.727344 29295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "d8d12730d746408296e1831de8e6607b"
format_stamp: "Formatted at 2026-08-07 19:36:09 on dist-test-slave-8w9v"
server_key: "376abeece54d8dcca0593dc67e58f53c"
server_key_iv: "01283b71e45ff372045055bcb0f0e437"
server_key_version: "encryptionkey@0"
I20260807 19:36:09.732681 29295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 19:36:09.736390 29310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:09.737763 29295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:09.737946 29295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "d8d12730d746408296e1831de8e6607b"
format_stamp: "Formatted at 2026-08-07 19:36:09 on dist-test-slave-8w9v"
server_key: "376abeece54d8dcca0593dc67e58f53c"
server_key_iv: "01283b71e45ff372045055bcb0f0e437"
server_key_version: "encryptionkey@0"
I20260807 19:36:09.738183 29295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:09.761440 29295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:09.762362 29295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:09.762703 29295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:09.782670 29295 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:35967
I20260807 19:36:09.782696 29365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:35967 every 8 connection(s)
I20260807 19:36:09.784175 29295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:09.788179 29366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:09.788528 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29295
I20260807 19:36:09.788717 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:09.789211 9058 external_mini_cluster.cc:1428] Setting key 1d4094c6cf67a7e68a7317ec5472df16
I20260807 19:36:09.804044 29366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: Bootstrap starting.
I20260807 19:36:09.806237 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:09.791202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50292 (local address 127.8.216.190:35967)
0807 19:36:09.792433 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:09.792453 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:09.792511 (+ 58us) server_negotiation.cc:407] Connection header received
0807 19:36:09.795421 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:09.795454 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:09.796175 (+ 721us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:09.796677 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:09.797246 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:09.799172 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:09.800770 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:09.801911 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:09.802093 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:09.803284 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:09.803295 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:09.803314 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:09.804074 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:09.804201 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:09.804229 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:09.804356 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:09.804555 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 19:36:09.804892 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":383,"threads_started":1}
I20260807 19:36:09.807926 29366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:09.809067 29366 log.cc:824] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:09.811725 29366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: No bootstrap required, opened a new log
I20260807 19:36:09.819471 29366 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } }
I20260807 19:36:09.819948 29366 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:09.820065 29366 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d12730d746408296e1831de8e6607b, State: Initialized, Role: FOLLOWER
I20260807 19:36:09.820684 29366 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } }
I20260807 19:36:09.820959 29366 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:09.821116 29366 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:09.821290 29366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:09.823658 29366 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } }
I20260807 19:36:09.824276 29366 leader_election.cc:302] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8d12730d746408296e1831de8e6607b; no voters:
I20260807 19:36:09.824887 29366 leader_election.cc:288] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:09.825037 29372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:09.825842 29372 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [term 1 LEADER]: Becoming Leader. State: Replica: d8d12730d746408296e1831de8e6607b, State: Running, Role: LEADER
I20260807 19:36:09.826388 29372 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } }
I20260807 19:36:09.827215 29366 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:09.829577 29373 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8d12730d746408296e1831de8e6607b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } } }
I20260807 19:36:09.829555 29374 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d12730d746408296e1831de8e6607b. Latest consensus state: current_term: 1 leader_uuid: "d8d12730d746408296e1831de8e6607b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d12730d746408296e1831de8e6607b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 35967 } } }
I20260807 19:36:09.830029 29373 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:09.830065 29374 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:09.832883 29378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:09.838335 29378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:09.847355 29378 catalog_manager.cc:1347] Generated new cluster ID: 557e568f003a42fda03dd9d1239f1496
I20260807 19:36:09.847494 29378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:09.870288 29378 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:09.871671 29378 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:09.885673 29378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: Generated new TSK 0
I20260807 19:36:09.886942 29378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:09.903359 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--builtin_ntp_servers=127.8.216.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:10.269210 29391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:10.269522 29391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:10.269659 29391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:10.277460 29391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:10.277601 29391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:10.277786 29391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:36:10.287451 29391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:10.288854 29391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:10.290704 29391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:10.304636 29400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:10.304646 29398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:10.304723 29397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:10.305576 29391 server_base.cc:1034] running on GCE node
I20260807 19:36:10.306206 29391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:10.307247 29391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:10.308480 29391 hybrid_clock.cc:648] HybridClock initialized: now 1786131370308434 us; error 61 us; skew 500 ppm
I20260807 19:36:10.308810 29391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:10.311259 29391 webserver.cc:460] Webserver started at http://127.8.216.129:35219/ using document root <none> and password file <none>
I20260807 19:36:10.312011 29391 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:10.312166 29391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:10.312541 29391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:10.315629 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "8029d21da9ca4e0c993636d892a8e39e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "3b8054bcdf82eee10a7aff69619d304e"
server_key_iv: "4a8fd8b12f0d9764ab1c7441cf634f22"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.316543 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "8029d21da9ca4e0c993636d892a8e39e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "3b8054bcdf82eee10a7aff69619d304e"
server_key_iv: "4a8fd8b12f0d9764ab1c7441cf634f22"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.321877 29391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:36:10.325387 29406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:10.326731 29391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:10.326908 29391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "8029d21da9ca4e0c993636d892a8e39e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "3b8054bcdf82eee10a7aff69619d304e"
server_key_iv: "4a8fd8b12f0d9764ab1c7441cf634f22"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.327167 29391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:10.344411 29391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:10.345278 29391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:10.345608 29391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:10.346597 29391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:10.348078 29391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:10.348248 29391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 19:36:10.348379 29391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:10.348477 29391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:10.382108 29391 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:34425
I20260807 19:36:10.382155 29524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:34425 every 8 connection(s)
I20260807 19:36:10.383471 29391 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:36:10.387698 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29391
I20260807 19:36:10.387897 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:36:10.388499 9058 external_mini_cluster.cc:1428] Setting key 11aa7e96f5a8c4cb2050d5434bb71a64
I20260807 19:36:10.391741 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--builtin_ntp_servers=127.8.216.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:10.398829 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:10.386385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:48797 (local address 127.8.216.190:35967)
0807 19:36:10.387064 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:10.387069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:10.388570 (+ 1501us) server_negotiation.cc:407] Connection header received
0807 19:36:10.390913 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:10.390917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:10.390995 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:10.391085 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:10.392590 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.393278 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.395785 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.396265 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.396362 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:10.398049 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:10.398054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:10.398057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:10.398075 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:10.398115 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:10.398126 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:10.398196 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:10.398474 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 19:36:10.398559 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":305,"threads_started":1}
I20260807 19:36:10.400398 29528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:10.386922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35967 (local address 127.8.216.129:48797)
0807 19:36:10.388373 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:10.388419 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:10.390592 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:10.391371 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:10.391384 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:10.391511 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:10.392410 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:10.392439 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.393443 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.393452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:10.395629 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:10.395648 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.396498 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.396507 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:10.396862 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:10.397664 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:10.397670 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:10.397727 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:10.397847 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:10.398226 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:10.398241 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:10.398252 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:36:10.398451 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 19:36:10.398913 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":341,"threads_started":1}
I20260807 19:36:10.402083 29525 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35967
I20260807 19:36:10.402405 29525 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:10.403234 29525 heartbeater.cc:511] Master 127.8.216.190:35967 requested a full tablet report, sending...
I20260807 19:36:10.405395 29329 ts_manager.cc:194] Registered new tserver with Master: 8029d21da9ca4e0c993636d892a8e39e (127.8.216.129:34425)
I20260807 19:36:10.407506 29329 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.129:48797
W20260807 19:36:10.764158 29529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:10.764503 29529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:10.764614 29529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:10.773011 29529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:10.773151 29529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:10.773269 29529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:36:10.783754 29529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:10.785367 29529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:10.787401 29529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:10.801627 29536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:10.801776 29534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:10.801779 29538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:10.802958 29529 server_base.cc:1034] running on GCE node
I20260807 19:36:10.803669 29529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:10.804857 29529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:10.806121 29529 hybrid_clock.cc:648] HybridClock initialized: now 1786131370806070 us; error 42 us; skew 500 ppm
I20260807 19:36:10.806473 29529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:10.809003 29529 webserver.cc:460] Webserver started at http://127.8.216.130:42469/ using document root <none> and password file <none>
I20260807 19:36:10.809777 29529 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:10.809924 29529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:10.810276 29529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:10.813424 29529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "d1979c6d427c48d89ecaece373a53e5e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "9b2427938d3f7920a08b32171f9b9d32"
server_key_iv: "f26a7f8a8cffcc2699134ac9c7719006"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.814261 29529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "d1979c6d427c48d89ecaece373a53e5e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "9b2427938d3f7920a08b32171f9b9d32"
server_key_iv: "f26a7f8a8cffcc2699134ac9c7719006"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.820443 29529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:36:10.824476 29544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:10.825964 29529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:10.826200 29529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "d1979c6d427c48d89ecaece373a53e5e"
format_stamp: "Formatted at 2026-08-07 19:36:10 on dist-test-slave-8w9v"
server_key: "9b2427938d3f7920a08b32171f9b9d32"
server_key_iv: "f26a7f8a8cffcc2699134ac9c7719006"
server_key_version: "encryptionkey@0"
I20260807 19:36:10.826491 29529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:10.857743 29529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:10.858724 29529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:10.859068 29529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:10.860217 29529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:10.861937 29529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:10.862071 29529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:10.862221 29529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:10.862313 29529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:10.900076 29529 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:36511
I20260807 19:36:10.900204 29672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:36511 every 8 connection(s)
I20260807 19:36:10.901620 29529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:36:10.908293 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29529
I20260807 19:36:10.908483 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:36:10.909075 9058 external_mini_cluster.cc:1428] Setting key b10e0db9a715530a8aa1183d35b1b718
I20260807 19:36:10.912556 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--builtin_ntp_servers=127.8.216.148:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:10.916745 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:10.904308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:48857 (local address 127.8.216.190:35967)
0807 19:36:10.904777 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:10.904782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:10.905953 (+ 1171us) server_negotiation.cc:407] Connection header received
0807 19:36:10.907963 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:10.907968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:10.908057 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:10.908237 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:10.909769 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.910632 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.913082 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.913834 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.914011 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:10.915851 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:10.915856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:10.915859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:10.915880 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:10.915956 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:10.915979 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:10.916053 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:10.916383 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:36:10.916492 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":257,"threads_started":1}
I20260807 19:36:10.918329 29677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:10.904825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35967 (local address 127.8.216.130:48857)
0807 19:36:10.905758 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:10.905802 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:10.907584 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:10.908525 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:10.908537 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:10.908670 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:10.909552 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:10.909579 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.910809 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.910814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:10.912927 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:10.912948 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:10.914182 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:10.914194 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:10.914594 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:10.915418 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:10.915424 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:10.915484 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:10.915622 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:10.916087 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:10.916102 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:10.916114 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:36:10.916346 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 19:36:10.916821 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":288,"threads_started":1}
I20260807 19:36:10.919938 29674 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35967
I20260807 19:36:10.920320 29674 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:10.921381 29674 heartbeater.cc:511] Master 127.8.216.190:35967 requested a full tablet report, sending...
I20260807 19:36:10.923166 29329 ts_manager.cc:194] Registered new tserver with Master: d1979c6d427c48d89ecaece373a53e5e (127.8.216.130:36511)
I20260807 19:36:10.924052 29329 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.130:48857
W20260807 19:36:11.271170 29678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:11.271500 29678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:11.271652 29678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:11.279287 29678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:11.279414 29678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:11.279597 29678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:36:11.290009 29678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:35967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:11.291428 29678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:11.293324 29678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:11.307039 29686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:11.307090 29684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:11.307094 29683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:11.309734 29678 server_base.cc:1034] running on GCE node
I20260807 19:36:11.310395 29678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:11.311436 29678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:11.312667 29678 hybrid_clock.cc:648] HybridClock initialized: now 1786131371312625 us; error 55 us; skew 500 ppm
I20260807 19:36:11.312999 29678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:11.315371 29678 webserver.cc:460] Webserver started at http://127.8.216.131:37447/ using document root <none> and password file <none>
I20260807 19:36:11.316180 29678 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:11.316314 29678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:11.316680 29678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:11.319836 29678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "9a7b7950d771480cb6ef3038beb204a9"
format_stamp: "Formatted at 2026-08-07 19:36:11 on dist-test-slave-8w9v"
server_key: "c8fa8b2646aaded5ee853b5907982d2e"
server_key_iv: "c9d2eb82243569b394a3c097055884b1"
server_key_version: "encryptionkey@0"
I20260807 19:36:11.320735 29678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "9a7b7950d771480cb6ef3038beb204a9"
format_stamp: "Formatted at 2026-08-07 19:36:11 on dist-test-slave-8w9v"
server_key: "c8fa8b2646aaded5ee853b5907982d2e"
server_key_iv: "c9d2eb82243569b394a3c097055884b1"
server_key_version: "encryptionkey@0"
I20260807 19:36:11.326531 29678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:36:11.330449 29692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:11.331969 29678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:36:11.332206 29678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "9a7b7950d771480cb6ef3038beb204a9"
format_stamp: "Formatted at 2026-08-07 19:36:11 on dist-test-slave-8w9v"
server_key: "c8fa8b2646aaded5ee853b5907982d2e"
server_key_iv: "c9d2eb82243569b394a3c097055884b1"
server_key_version: "encryptionkey@0"
I20260807 19:36:11.332461 29678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:11.354923 29678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:11.355949 29678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:11.356375 29678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:11.357514 29678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:11.359171 29678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:11.359305 29678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:11.359431 29678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:11.359519 29678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:11.396456 29678 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:40727
I20260807 19:36:11.396509 29812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:40727 every 8 connection(s)
I20260807 19:36:11.397814 29678 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:36:11.398535 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29678
I20260807 19:36:11.398736 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:36:11.399366 9058 external_mini_cluster.cc:1428] Setting key e2d0a10c6c80f4ffc4af11732db20704
I20260807 19:36:11.410750 29525 heartbeater.cc:503] Master 127.8.216.190:35967 was elected leader, sending a full tablet report...
I20260807 19:36:11.412254 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.400998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:56911 (local address 127.8.216.190:35967)
0807 19:36:11.401218 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.401226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.402923 (+ 1697us) server_negotiation.cc:407] Connection header received
0807 19:36:11.404970 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.404975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.405060 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.405160 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:11.406347 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.407087 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.408896 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.409327 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.409458 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.411198 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:11.411204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:11.411209 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:11.411234 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:11.411277 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:11.411296 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:11.411404 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.411740 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.411852 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:36:11.413558 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.401626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:35967 (local address 127.8.216.131:56911)
0807 19:36:11.402752 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.402800 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.404693 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.405435 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.405445 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.405543 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:11.406154 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.406172 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.407283 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.407293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.408728 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.408744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.409578 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.409583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.409802 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.410716 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:11.410720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:11.410782 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:11.410949 (+ 167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:11.411430 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:11.411443 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:11.411474 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.411716 (+ 242us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.412209 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":376,"threads_started":1}
I20260807 19:36:11.415012 29813 heartbeater.cc:347] Connected to a master server at 127.8.216.190:35967
I20260807 19:36:11.415342 29813 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:11.416211 29813 heartbeater.cc:511] Master 127.8.216.190:35967 requested a full tablet report, sending...
I20260807 19:36:11.417732 29329 ts_manager.cc:194] Registered new tserver with Master: 9a7b7950d771480cb6ef3038beb204a9 (127.8.216.131:40727)
I20260807 19:36:11.418632 29329 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.131:56911
I20260807 19:36:11.428515 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:36:11.440261 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.433372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.8.216.190:35967)
0807 19:36:11.433611 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.433639 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.433936 (+ 297us) server_negotiation.cc:407] Connection header received
0807 19:36:11.434163 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.434169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.434249 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.434363 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:11.434919 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.435687 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.437498 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.437949 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.438102 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.439367 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:11.439372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:11.439375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:11.439395 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:11.439434 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:11.439446 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:11.439604 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.439808 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.439871 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:36:11.444902 29329 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:11.452215 29329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:11.477447 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.465602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:36511 (local address 127.0.0.1:56722)
0807 19:36:11.467073 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.467124 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.467281 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.470342 (+ 3061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.470347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.470380 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:11.470998 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.471010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.472643 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.472649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.474824 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.474837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.476349 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.476357 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.476583 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.476639 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.476794 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.477020 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":3,"thread_start_us":1042,"threads_started":1}
I20260807 19:36:11.478309 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.465669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.8.216.130:36511)
0807 19:36:11.467891 (+ 2222us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.467902 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.467934 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:36:11.468042 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.468051 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.468356 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.470079 (+ 1723us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:11.471314 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.472478 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.475023 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.476151 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.476306 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.476571 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.476855 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.477050 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2045,"thread_start_us":1941,"threads_started":1}
I20260807 19:36:11.478731 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.466153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:40727 (local address 127.0.0.1:60226)
0807 19:36:11.468996 (+ 2843us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.469020 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.469172 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.470227 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.470235 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.470264 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:11.470990 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.471009 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.473098 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.473104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.475775 (+ 2671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.475790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.477860 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.477869 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.478065 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.478115 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.478195 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.478354 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2684,"mutex_wait_us":58,"thread_start_us":2544,"threads_started":1}
I20260807 19:36:11.479761 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.467542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60054)
0807 19:36:11.469775 (+ 2233us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.469795 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.469908 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.470618 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.470626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.470649 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:11.471321 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.471333 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.473283 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.473289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.475739 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.475756 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.478955 (+ 3199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.478962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.479129 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.479153 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.479264 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.479416 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2107,"mutex_wait_us":4,"thread_start_us":2779,"threads_started":1}
I20260807 19:36:11.479882 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.468086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.8.216.129:34425)
0807 19:36:11.469335 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.469343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.470043 (+ 700us) server_negotiation.cc:407] Connection header received
0807 19:36:11.470167 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.470175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.470410 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.470611 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:11.471526 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.473105 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.476271 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.478351 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.478521 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.478809 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.479322 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.479520 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":1985,"threads_started":1}
I20260807 19:36:11.482282 29597 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 (DEFAULT_TABLE table=test-table [id=be72ed834da0422abc9238c6b3d3b500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:11.482990 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.466956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.8.216.131:40727)
0807 19:36:11.468499 (+ 1543us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.468522 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.469307 (+ 785us) server_negotiation.cc:407] Connection header received
0807 19:36:11.469418 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.469428 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.469670 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.469873 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:11.471269 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.472930 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.475997 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.477410 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.478959 (+ 1549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.479265 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.479340 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.482584 (+ 3244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1341,"thread_start_us":1165,"threads_started":1}
I20260807 19:36:11.483997 29597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1082989f154c179d234ba2bc5cd6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:11.483661 29457 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 (DEFAULT_TABLE table=test-table [id=be72ed834da0422abc9238c6b3d3b500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:11.485194 29457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1082989f154c179d234ba2bc5cd6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:11.486788 29744 tablet_service.cc:1467] Processing CreateTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 (DEFAULT_TABLE table=test-table [id=be72ed834da0422abc9238c6b3d3b500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:11.488314 29744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f1082989f154c179d234ba2bc5cd6b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:11.495900 29833 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Bootstrap starting.
I20260807 19:36:11.500468 29834 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Bootstrap starting.
I20260807 19:36:11.501133 29833 tablet_bootstrap.cc:654] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:11.502537 29833 log.cc:824] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:11.503580 29835 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Bootstrap starting.
I20260807 19:36:11.504863 29834 tablet_bootstrap.cc:654] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:11.506493 29834 log.cc:824] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:11.507472 29835 tablet_bootstrap.cc:654] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:11.507536 29833 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: No bootstrap required, opened a new log
I20260807 19:36:11.507843 29833 ts_tablet_manager.cc:1397] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:36:11.508808 29835 log.cc:824] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:11.509440 29834 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: No bootstrap required, opened a new log
I20260807 19:36:11.509743 29834 ts_tablet_manager.cc:1397] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:36:11.512285 29835 tablet_bootstrap.cc:492] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: No bootstrap required, opened a new log
I20260807 19:36:11.512583 29835 ts_tablet_manager.cc:1397] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:36:11.518350 29834 raft_consensus.cc:348] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.518890 29834 raft_consensus.cc:374] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:11.518994 29834 raft_consensus.cc:729] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8029d21da9ca4e0c993636d892a8e39e, State: Initialized, Role: FOLLOWER
I20260807 19:36:11.519186 29833 raft_consensus.cc:348] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.519663 29833 raft_consensus.cc:374] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:11.519778 29833 raft_consensus.cc:729] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1979c6d427c48d89ecaece373a53e5e, State: Initialized, Role: FOLLOWER
I20260807 19:36:11.519760 29834 consensus_queue.cc:260] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.520429 29833 consensus_queue.cc:260] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.522984 29834 ts_tablet_manager.cc:1428] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:36:11.523934 29674 heartbeater.cc:503] Master 127.8.216.190:35967 was elected leader, sending a full tablet report...
I20260807 19:36:11.524657 29833 ts_tablet_manager.cc:1428] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Time spent starting tablet: real 0.017s user 0.019s sys 0.000s
I20260807 19:36:11.525103 29835 raft_consensus.cc:348] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.525610 29835 raft_consensus.cc:374] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:11.525722 29835 raft_consensus.cc:729] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a7b7950d771480cb6ef3038beb204a9, State: Initialized, Role: FOLLOWER
I20260807 19:36:11.526382 29835 consensus_queue.cc:260] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.528667 29813 heartbeater.cc:503] Master 127.8.216.190:35967 was elected leader, sending a full tablet report...
I20260807 19:36:11.529314 29835 ts_tablet_manager.cc:1428] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260807 19:36:11.607157 29841 raft_consensus.cc:482] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:11.607554 29841 raft_consensus.cc:504] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.609625 29841 leader_election.cc:288] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1979c6d427c48d89ecaece373a53e5e (127.8.216.130:36511), 8029d21da9ca4e0c993636d892a8e39e (127.8.216.129:34425)
I20260807 19:36:11.615715 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.610274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:44011 (local address 127.8.216.130:36511)
0807 19:36:11.610505 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.610512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.610538 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:36:11.610632 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.610639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.610747 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.610909 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:11.611509 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.612317 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.613977 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.614760 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.614932 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.615009 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.615291 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.615422 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:36:11.615710 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.610047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:36511 (local address 127.8.216.131:44011)
0807 19:36:11.610290 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.610315 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.610491 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.611149 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.611154 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.611178 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:11.611341 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.611349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.612456 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.612463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.613805 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.613817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.614919 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.614925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.615109 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.615134 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.615240 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.615386 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:36:11.616622 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.610838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:46483 (local address 127.8.216.129:34425)
0807 19:36:11.611062 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.611067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.611386 (+ 319us) server_negotiation.cc:407] Connection header received
0807 19:36:11.611597 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.611602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.611685 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.611793 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:11.612440 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.613124 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.614830 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.615635 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.615817 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.615886 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.616252 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.616383 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:36:11.616655 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.610471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.8.216.131:46483)
0807 19:36:11.611261 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.611276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:11.611432 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:11.611810 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:11.611814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:11.611837 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:11.612251 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.612261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.613289 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.613298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.614634 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:11.614646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.615818 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.615824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:11.616055 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.616095 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:36:11.616223 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 19:36:11.616350 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":445,"threads_started":1}
I20260807 19:36:11.616994 29622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1082989f154c179d234ba2bc5cd6b9" candidate_uuid: "9a7b7950d771480cb6ef3038beb204a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1979c6d427c48d89ecaece373a53e5e" is_pre_election: true
I20260807 19:36:11.617575 29622 raft_consensus.cc:2454] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a7b7950d771480cb6ef3038beb204a9 in term 0.
I20260807 19:36:11.617738 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1082989f154c179d234ba2bc5cd6b9" candidate_uuid: "9a7b7950d771480cb6ef3038beb204a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8029d21da9ca4e0c993636d892a8e39e" is_pre_election: true
I20260807 19:36:11.618335 29478 raft_consensus.cc:2454] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a7b7950d771480cb6ef3038beb204a9 in term 0.
I20260807 19:36:11.618351 29693 leader_election.cc:302] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a7b7950d771480cb6ef3038beb204a9, d1979c6d427c48d89ecaece373a53e5e; no voters:
I20260807 19:36:11.618796 29841 raft_consensus.cc:2781] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:11.618991 29841 raft_consensus.cc:482] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:11.619119 29841 raft_consensus.cc:3037] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:11.621564 29841 raft_consensus.cc:504] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.622231 29841 leader_election.cc:288] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [CANDIDATE]: Term 1 election: Requested vote from peers d1979c6d427c48d89ecaece373a53e5e (127.8.216.130:36511), 8029d21da9ca4e0c993636d892a8e39e (127.8.216.129:34425)
I20260807 19:36:11.622999 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1082989f154c179d234ba2bc5cd6b9" candidate_uuid: "9a7b7950d771480cb6ef3038beb204a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8029d21da9ca4e0c993636d892a8e39e"
I20260807 19:36:11.622991 29622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f1082989f154c179d234ba2bc5cd6b9" candidate_uuid: "9a7b7950d771480cb6ef3038beb204a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1979c6d427c48d89ecaece373a53e5e"
I20260807 19:36:11.623241 29622 raft_consensus.cc:3037] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:11.623237 29478 raft_consensus.cc:3037] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:11.625754 29478 raft_consensus.cc:2454] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a7b7950d771480cb6ef3038beb204a9 in term 1.
I20260807 19:36:11.625792 29622 raft_consensus.cc:2454] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a7b7950d771480cb6ef3038beb204a9 in term 1.
I20260807 19:36:11.626235 29693 leader_election.cc:302] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a7b7950d771480cb6ef3038beb204a9, d1979c6d427c48d89ecaece373a53e5e; no voters:
I20260807 19:36:11.626578 29841 raft_consensus.cc:2781] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:11.627168 29841 raft_consensus.cc:686] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 1 LEADER]: Becoming Leader. State: Replica: 9a7b7950d771480cb6ef3038beb204a9, State: Running, Role: LEADER
I20260807 19:36:11.627980 29841 consensus_queue.cc:237] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } }
I20260807 19:36:11.633545 29329 catalog_manager.cc:5495] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a7b7950d771480cb6ef3038beb204a9 (127.8.216.131). New cstate: current_term: 1 leader_uuid: "9a7b7950d771480cb6ef3038beb204a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a7b7950d771480cb6ef3038beb204a9" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 40727 } health_report { overall_health: HEALTHY } } }
W20260807 19:36:11.636523 29526 tablet.cc:2369] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:11.649879 29814 tablet.cc:2369] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:11.653815 29675 tablet.cc:2369] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:11.673313 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.667757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.8.216.131:40727)
0807 19:36:11.667947 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.667952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.667971 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:36:11.668062 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.668065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.668226 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.668363 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:11.668751 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.669520 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.670958 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.671410 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.671514 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.672027 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:11.672662 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:11.672800 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.672957 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.673027 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:36:11.682031 29622 raft_consensus.cc:1261] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 1 FOLLOWER]: Refusing update from remote peer 9a7b7950d771480cb6ef3038beb204a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:11.682063 29478 raft_consensus.cc:1261] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 1 FOLLOWER]: Refusing update from remote peer 9a7b7950d771480cb6ef3038beb204a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:11.683101 29841 consensus_queue.cc:1035] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8029d21da9ca4e0c993636d892a8e39e" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:11.683454 29843 consensus_queue.cc:1035] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1979c6d427c48d89ecaece373a53e5e" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 36511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:11.696328 29846 mvcc.cc:205] Tried to move back new op lower bound from 7315994098398715904 to 7315994098192596992. Current Snapshot: MvccSnapshot[applied={T|T < 7315994098398715904}]
I20260807 19:36:11.699704 29847 mvcc.cc:205] Tried to move back new op lower bound from 7315994098398715904 to 7315994098192596992. Current Snapshot: MvccSnapshot[applied={T|T < 7315994098398715904}]
I20260807 19:36:11.714195 29329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:11.714497 29329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:11.718863 29329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8d12730d746408296e1831de8e6607b: Sending DeleteTablet for 3 replicas of tablet 9f1082989f154c179d234ba2bc5cd6b9
I20260807 19:36:11.720408 29744 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:11 UTC) from {username='slave'} at 127.0.0.1:60226
I20260807 19:36:11.720468 29457 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:11 UTC) from {username='slave'} at 127.0.0.1:60054
I20260807 19:36:11.720681 29597 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f1082989f154c179d234ba2bc5cd6b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:11 UTC) from {username='slave'} at 127.0.0.1:56722
I20260807 19:36:11.721426 29856 tablet_replica.cc:330] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: stopping tablet replica
I20260807 19:36:11.721951 29856 raft_consensus.cc:2229] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:11.722554 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29391
I20260807 19:36:11.722736 29856 raft_consensus.cc:2258] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:11.725023 29856 ts_tablet_manager.cc:1905] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:11.733712 29856 ts_tablet_manager.cc:1918] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:11.733937 29856 log.cc:1196] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/9f1082989f154c179d234ba2bc5cd6b9
I20260807 19:36:11.734696 29856 ts_tablet_manager.cc:1939] T 9f1082989f154c179d234ba2bc5cd6b9 P 9a7b7950d771480cb6ef3038beb204a9: Deleting consensus metadata
I20260807 19:36:11.737032 29312 catalog_manager.cc:4841] TS 9a7b7950d771480cb6ef3038beb204a9 (127.8.216.131:40727): tablet 9f1082989f154c179d234ba2bc5cd6b9 (table test-table [id=be72ed834da0422abc9238c6b3d3b500]) successfully deleted
I20260807 19:36:11.741696 29858 tablet_replica.cc:330] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: stopping tablet replica
I20260807 19:36:11.742084 29857 tablet_replica.cc:330] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: stopping tablet replica
I20260807 19:36:11.742264 29858 raft_consensus.cc:2229] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:11.742660 29857 raft_consensus.cc:2229] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:11.742911 29858 raft_consensus.cc:2258] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:11.743294 29857 raft_consensus.cc:2258] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:11.745601 29857 ts_tablet_manager.cc:1905] T 9f1082989f154c179d234ba2bc5cd6b9 P 8029d21da9ca4e0c993636d892a8e39e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:11.748909 29858 ts_tablet_manager.cc:1905] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:11.751644 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.725297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.8.216.129:34425)
0807 19:36:11.725497 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.725505 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.740825 (+ 15320us) server_negotiation.cc:407] Connection header received
0807 19:36:11.741079 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.741088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.741262 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.741381 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:11.742114 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.743344 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.745331 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.745958 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.746064 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.748909 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:11.748955 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:11.748975 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:11.750211 (+ 1236us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:11.750308 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:11.750364 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:11.750489 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.750912 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.751126 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:36:11.753172 29520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:11.759666 29858 ts_tablet_manager.cc:1918] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:11.759969 29858 log.cc:1196] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/9f1082989f154c179d234ba2bc5cd6b9
I20260807 19:36:11.760723 29858 ts_tablet_manager.cc:1939] T 9f1082989f154c179d234ba2bc5cd6b9 P d1979c6d427c48d89ecaece373a53e5e: Deleting consensus metadata
I20260807 19:36:11.763348 29311 catalog_manager.cc:4841] TS d1979c6d427c48d89ecaece373a53e5e (127.8.216.130:36511): tablet 9f1082989f154c179d234ba2bc5cd6b9 (table test-table [id=be72ed834da0422abc9238c6b3d3b500]) successfully deleted
I20260807 19:36:11.873194 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29391
I20260807 19:36:11.895599 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29529
W20260807 19:36:11.895659 29311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:34425 (error 108)
I20260807 19:36:11.898269 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.897776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60074)
0807 19:36:11.897960 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.898088 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 19:36:11.899286 29311 catalog_manager.cc:4584] TS 8029d21da9ca4e0c993636d892a8e39e (127.8.216.129:34425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f1082989f154c179d234ba2bc5cd6b9: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
I20260807 19:36:11.904337 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.896870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.8.216.130:36511)
0807 19:36:11.897231 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:11.897236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:11.897251 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:36:11.897314 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:11.897319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:11.897422 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:11.897503 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:11.898033 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.898949 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.900845 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:11.901301 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:11.901480 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:11.902832 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:11.902840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:11.902853 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:11.903465 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:11.903517 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:11.903538 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:11.903674 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:11.903820 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 19:36:11.903936 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260807 19:36:11.905068 29668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:11.953299 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.952674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60088)
0807 19:36:11.952956 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.953077 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 19:36:11.954571 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:11.954146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60102)
0807 19:36:11.954324 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:11.954415 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:36:11.999594 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29529
I20260807 19:36:12.015467 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.014818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60112)
0807 19:36:12.015111 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:12.015236 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:36:12.016898 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.016441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60122)
0807 19:36:12.016636 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:12.016733 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:36:12.023165 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29678
I20260807 19:36:12.031772 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.024008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.8.216.131:40727)
0807 19:36:12.024232 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:12.024237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:12.024309 (+ 72us) server_negotiation.cc:407] Connection header received
0807 19:36:12.024523 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:12.024527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:12.024611 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:12.024727 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:12.025308 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:12.026172 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:12.028216 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:12.028693 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:12.028849 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:12.030232 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:12.030244 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:12.030259 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:12.030888 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:12.030953 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:12.030973 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:12.031071 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:12.031265 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 19:36:12.031358 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:36:12.032536 29808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:12.097898 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.097329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60132)
0807 19:36:12.097603 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:12.097708 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 19:36:12.099090 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.098669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60142)
0807 19:36:12.098892 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:12.098975 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 19:36:12.160661 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29678
I20260807 19:36:12.185539 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29295
I20260807 19:36:12.186811 29361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:12.301169 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29295
I20260807 19:36:12.301249 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:12.300686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:34425 (local address 127.0.0.1:60148)
0807 19:36:12.300916 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:12.301014 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:34425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
2026-08-07T19:36:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3074 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:12 dist-test-slave-8w9v krb5kdc[29867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:12 dist-test-slave-8w9v krb5kdc[29867](info): set up 2 sockets
Aug 07 19:36:12 dist-test-slave-8w9v krb5kdc[29867](info): commencing operation
krb5kdc: starting...
W20260807 19:36:13.495518 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:13 dist-test-slave-8w9v krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:13.832857 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33799
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:14.189515 29881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:14.189818 29881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:14.189913 29881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:14.197299 29881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:14.197413 29881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:14.197530 29881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:14.197636 29881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:14.197722 29881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:14.207798 29881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:14.209264 29881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:14.210644 29881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:14.223232 29886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:14.223270 29889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:14.223270 29887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:14.224673 29881 server_base.cc:1034] running on GCE node
I20260807 19:36:14.225559 29881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:14.227057 29881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:14.228308 29881 hybrid_clock.cc:648] HybridClock initialized: now 1786131374228263 us; error 58 us; skew 500 ppm
I20260807 19:36:14.228694 29881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:14 dist-test-slave-8w9v krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:14.237250 29881 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:14.239002 29881 webserver.cc:460] Webserver started at http://127.8.216.190:44039/ using document root <none> and password file <none>
I20260807 19:36:14.239775 29881 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:14.239941 29881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:14.240360 29881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:14.243680 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "5ca39f87575f458c83407d7970be7019"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "6e5587d9bc3bf00c6c2a1e7f18295fa1"
server_key_iv: "9d1cf18444b888887f0910ec317c6c1c"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.244726 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "5ca39f87575f458c83407d7970be7019"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "6e5587d9bc3bf00c6c2a1e7f18295fa1"
server_key_iv: "9d1cf18444b888887f0910ec317c6c1c"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.250811 29881 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260807 19:36:14.254715 29897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:14.256286 29881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:36:14.256505 29881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "5ca39f87575f458c83407d7970be7019"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "6e5587d9bc3bf00c6c2a1e7f18295fa1"
server_key_iv: "9d1cf18444b888887f0910ec317c6c1c"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.256703 29881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:14.278726 29881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:14.283416 29881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:14.283754 29881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:14.305416 29881 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33799
I20260807 19:36:14.305454 29950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33799 every 8 connection(s)
I20260807 19:36:14.306954 29881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:14.310978 29951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:14.316579 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29881
I20260807 19:36:14.316780 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:14.317304 9058 external_mini_cluster.cc:1428] Setting key 447fadf39611da26460034553203758b
I20260807 19:36:14.323177 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: Bootstrap starting.
I20260807 19:36:14.326719 29951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:14.327800 29951 log.cc:824] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:14.330983 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: No bootstrap required, opened a new log
Aug 07 19:36:14 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:14.341104 29951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } }
I20260807 19:36:14.341539 29951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:14.341686 29951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca39f87575f458c83407d7970be7019, State: Initialized, Role: FOLLOWER
I20260807 19:36:14.341744 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:14.319467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.8.216.190:33799)
0807 19:36:14.320383 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:14.320414 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:14.320464 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:36:14.322294 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:14.322319 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:14.322742 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:14.323172 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:14.324959 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:14.326283 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:14.327797 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:14.328677 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:14.328893 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:14.333661 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:14.333672 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:14.333688 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:14.333731 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:14.337130 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:14.338069 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:14.338076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:14.338082 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:14.338266 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:14.338720 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:14.338723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:14.338725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:14.339312 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:14.339576 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:14.340039 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 19:36:14.340581 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":380,"threads_started":1}
I20260807 19:36:14.342362 29951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } }
I20260807 19:36:14.342633 29951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:14.342787 29951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:14.343005 29951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:14.345521 29951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } }
I20260807 19:36:14.346024 29951 leader_election.cc:302] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ca39f87575f458c83407d7970be7019; no voters:
I20260807 19:36:14.346648 29951 leader_election.cc:288] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:14.346813 29956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:14.347666 29956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [term 1 LEADER]: Becoming Leader. State: Replica: 5ca39f87575f458c83407d7970be7019, State: Running, Role: LEADER
I20260807 19:36:14.348342 29956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } }
I20260807 19:36:14.348958 29951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:14.351279 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ca39f87575f458c83407d7970be7019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } } }
I20260807 19:36:14.351536 29958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ca39f87575f458c83407d7970be7019. Latest consensus state: current_term: 1 leader_uuid: "5ca39f87575f458c83407d7970be7019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ca39f87575f458c83407d7970be7019" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33799 } } }
I20260807 19:36:14.351780 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:14.351850 29958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:14.354158 29963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:14.359724 29963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:14.374702 29963 catalog_manager.cc:1347] Generated new cluster ID: d11c741b2b8c408c985e69f990de664c
I20260807 19:36:14.374838 29963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:14.414657 29963 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:14.416571 29963 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:14.424527 29963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: Generated new TSK 0
I20260807 19:36:14.425441 29963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.216.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
Entry for principal kudu/127.8.216.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab.
I20260807 19:36:14.501272 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:14.861999 29979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:14.862356 29979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:14.862530 29979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:14.870277 29979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:14.870461 29979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:14.870838 29979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:36:14.881619 29979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.129.keytab
--principal=kudu/127.8.216.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:14.883090 29979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:14.884581 29979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:14.898615 29985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:14.898641 29984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:14.898766 29987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:14.899411 29979 server_base.cc:1034] running on GCE node
I20260807 19:36:14.900099 29979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:14.901191 29979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:14.902451 29979 hybrid_clock.cc:648] HybridClock initialized: now 1786131374902395 us; error 67 us; skew 500 ppm
I20260807 19:36:14.902803 29979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:14 dist-test-slave-8w9v krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:14.906250 29979 init.cc:377] Logged in from keytab as kudu/127.8.216.129@KRBTEST.COM (short username kudu)
I20260807 19:36:14.907871 29979 webserver.cc:460] Webserver started at http://127.8.216.129:34917/ using document root <none> and password file <none>
I20260807 19:36:14.908710 29979 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:14.908869 29979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:14.909212 29979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:14.912457 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "6a682669bdfe472d90526c897c82cc83"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "d00b37c1a1d954b3e03e520bb384b8d2"
server_key_iv: "0346f26cca6b7c4cbcce6a6cdc5bd448"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.913400 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "6a682669bdfe472d90526c897c82cc83"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "d00b37c1a1d954b3e03e520bb384b8d2"
server_key_iv: "0346f26cca6b7c4cbcce6a6cdc5bd448"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.918957 29979 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:36:14.922642 29994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:14.924535 29979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:36:14.924774 29979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "6a682669bdfe472d90526c897c82cc83"
format_stamp: "Formatted at 2026-08-07 19:36:14 on dist-test-slave-8w9v"
server_key: "d00b37c1a1d954b3e03e520bb384b8d2"
server_key_iv: "0346f26cca6b7c4cbcce6a6cdc5bd448"
server_key_version: "encryptionkey@0"
I20260807 19:36:14.924998 29979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:14.952538 29979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:14.956804 29979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:14.957072 29979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:14.958130 29979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:14.959494 29979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:14.959573 29979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:14.959662 29979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:14.959707 29979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:14.993954 29979 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:40397
I20260807 19:36:14.993999 30115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:40397 every 8 connection(s)
I20260807 19:36:14.995321 29979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:36:14.996946 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 29979
I20260807 19:36:14.997128 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:36:14.997720 9058 external_mini_cluster.cc:1428] Setting key fa211deb8bf37e99ca14782199ae92f8
Aug 07 19:36:15 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.129@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:15.017130 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:14.998250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:49731 (local address 127.8.216.190:33799)
0807 19:36:14.998756 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:14.998761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:14.999986 (+ 1225us) server_negotiation.cc:407] Connection header received
0807 19:36:15.002027 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:15.002033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:15.002128 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:15.002254 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:15.004009 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.004829 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.006449 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.006865 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.007008 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:15.010852 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:15.010857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:15.010861 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:15.010896 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:15.013689 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:15.014514 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:15.014520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:15.014523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:15.014666 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:15.015141 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:15.015148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:15.015151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:15.015474 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:15.015639 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:15.016485 (+ 846us) server_negotiation.cc:299] Negotiation successful
0807 19:36:15.016738 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":273,"threads_started":1}
I20260807 19:36:15.018296 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:14.998925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33799 (local address 127.8.216.129:49731)
0807 19:36:14.999797 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:14.999847 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:15.001734 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:15.002519 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:15.002528 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:15.003097 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:15.003812 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:15.003832 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.005008 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.005012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:15.006274 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:15.006287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.007128 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.007135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:15.007390 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:15.008172 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:15.008208 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:15.010584 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:15.013865 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:15.013875 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:15.013894 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:15.014377 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:15.014789 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:15.014794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:15.014798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:15.015019 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:15.015615 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:15.015624 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:15.016025 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 19:36:15.016378 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 19:36:15.017013 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":306,"threads_started":1}
I20260807 19:36:15.020080 30117 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33799
I20260807 19:36:15.020486 30117 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:15.021248 30117 heartbeater.cc:511] Master 127.8.216.190:33799 requested a full tablet report, sending...
I20260807 19:36:15.023825 29915 ts_manager.cc:194] Registered new tserver with Master: 6a682669bdfe472d90526c897c82cc83 (127.8.216.129:40397)
I20260807 19:36:15.026239 29915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.129@KRBTEST.COM'} at 127.8.216.129:49731
WARNING: no policy specified for kudu/127.8.216.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
Entry for principal kudu/127.8.216.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab.
I20260807 19:36:15.072059 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:15.431766 30123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:15.432062 30123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:15.432245 30123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:15.439703 30123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:15.439819 30123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:15.439941 30123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:36:15.450217 30123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.130.keytab
--principal=kudu/127.8.216.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:15.451594 30123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:15.452983 30123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:15.467139 30132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:15.467192 30128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:15.467217 30129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:15.468189 30123 server_base.cc:1034] running on GCE node
I20260807 19:36:15.468815 30123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:15.469781 30123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:15.470988 30123 hybrid_clock.cc:648] HybridClock initialized: now 1786131375470921 us; error 56 us; skew 500 ppm
I20260807 19:36:15.471329 30123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:15 dist-test-slave-8w9v krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:15.474607 30123 init.cc:377] Logged in from keytab as kudu/127.8.216.130@KRBTEST.COM (short username kudu)
I20260807 19:36:15.476409 30123 webserver.cc:460] Webserver started at http://127.8.216.130:39649/ using document root <none> and password file <none>
I20260807 19:36:15.477154 30123 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:15.477312 30123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:15.477706 30123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:15.480944 30123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "8f1dc04ea4374537850a94d9017897b5"
format_stamp: "Formatted at 2026-08-07 19:36:15 on dist-test-slave-8w9v"
server_key: "ab6db21e9767166503edbe952c4d1d28"
server_key_iv: "5249791453e0f4c6534393a74bea1496"
server_key_version: "encryptionkey@0"
I20260807 19:36:15.481797 30123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "8f1dc04ea4374537850a94d9017897b5"
format_stamp: "Formatted at 2026-08-07 19:36:15 on dist-test-slave-8w9v"
server_key: "ab6db21e9767166503edbe952c4d1d28"
server_key_iv: "5249791453e0f4c6534393a74bea1496"
server_key_version: "encryptionkey@0"
I20260807 19:36:15.487273 30123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:36:15.490895 30139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:15.492396 30123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:36:15.492590 30123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "8f1dc04ea4374537850a94d9017897b5"
format_stamp: "Formatted at 2026-08-07 19:36:15 on dist-test-slave-8w9v"
server_key: "ab6db21e9767166503edbe952c4d1d28"
server_key_iv: "5249791453e0f4c6534393a74bea1496"
server_key_version: "encryptionkey@0"
I20260807 19:36:15.492817 30123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:15.512933 30123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:15.517297 30123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:15.517622 30123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:15.518684 30123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:15.520231 30123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:15.520339 30123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:15.520506 30123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:15.520581 30123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:15.555604 30123 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:33463
I20260807 19:36:15.555637 30261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:33463 every 8 connection(s)
I20260807 19:36:15.557200 30123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:36:15.566396 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30123
I20260807 19:36:15.566577 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:36:15.567037 9058 external_mini_cluster.cc:1428] Setting key 81479834bd4d3c4f29c794bf06673702
Aug 07 19:36:15 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131375, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.130@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:15.579844 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:15.559948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:44115 (local address 127.8.216.190:33799)
0807 19:36:15.560483 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:15.560491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:15.561543 (+ 1052us) server_negotiation.cc:407] Connection header received
0807 19:36:15.563450 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:15.563455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:15.563525 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:15.563625 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:15.565205 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.565926 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.567659 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.568414 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.568554 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:15.572716 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:15.572725 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:15.572733 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:15.572782 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:15.576414 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:15.577150 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:15.577157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:15.577163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:15.577337 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:15.577767 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:15.577775 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:15.577781 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:15.578203 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:15.578364 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:15.579057 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 19:36:15.579330 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":261,"threads_started":1}
I20260807 19:36:15.580502 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:15.560560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33799 (local address 127.8.216.130:44115)
0807 19:36:15.561300 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:15.561408 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:15.563195 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:15.563842 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:15.563851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:15.564486 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:15.565051 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:15.565068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.566080 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.566083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:15.567510 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:15.567523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:15.568673 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:15.568682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:15.568919 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:15.569644 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:15.569685 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:15.572471 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:15.576556 (+ 4085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:15.576563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:15.576577 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:15.577012 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:15.577467 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:15.577470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:15.577472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:15.577647 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:15.578326 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:15.578332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:15.578682 (+ 350us) client_negotiation.cc:769] Sending connection context
0807 19:36:15.578911 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 19:36:15.579317 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":292,"threads_started":1}
I20260807 19:36:15.582070 30262 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33799
I20260807 19:36:15.582376 30262 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:15.583117 30262 heartbeater.cc:511] Master 127.8.216.190:33799 requested a full tablet report, sending...
I20260807 19:36:15.584780 29915 ts_manager.cc:194] Registered new tserver with Master: 8f1dc04ea4374537850a94d9017897b5 (127.8.216.130:33463)
I20260807 19:36:15.585632 29915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.130@KRBTEST.COM'} at 127.8.216.130:44115
WARNING: no policy specified for kudu/127.8.216.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
Entry for principal kudu/127.8.216.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab.
I20260807 19:36:15.641909 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:15.999917 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:16.000274 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:16.000488 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:16.008060 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:16.008311 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:16.008535 30270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:36:16.018889 30270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.131.keytab
--principal=kudu/127.8.216.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:33799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:16.020541 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:16.021927 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:36:16.030773 30117 heartbeater.cc:503] Master 127.8.216.190:33799 was elected leader, sending a full tablet report...
W20260807 19:36:16.036684 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:16.036631 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:16.036739 30279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:16.041872 30270 server_base.cc:1034] running on GCE node
I20260807 19:36:16.042711 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:16.043896 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:16.045166 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786131376045122 us; error 31 us; skew 500 ppm
I20260807 19:36:16.045560 30270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:16 dist-test-slave-8w9v krb5kdc[29867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:16.048993 30270 init.cc:377] Logged in from keytab as kudu/127.8.216.131@KRBTEST.COM (short username kudu)
I20260807 19:36:16.050717 30270 webserver.cc:460] Webserver started at http://127.8.216.131:45373/ using document root <none> and password file <none>
I20260807 19:36:16.051520 30270 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:16.051677 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:16.052017 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:16.055161 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "a0bb2d180aa145d287a98e080d0bc4f0"
format_stamp: "Formatted at 2026-08-07 19:36:16 on dist-test-slave-8w9v"
server_key: "328be6d90702f2d8f0a7ac0cb94b1ff7"
server_key_iv: "b93496f13954bd9df28d4d33610f373a"
server_key_version: "encryptionkey@0"
I20260807 19:36:16.056071 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "a0bb2d180aa145d287a98e080d0bc4f0"
format_stamp: "Formatted at 2026-08-07 19:36:16 on dist-test-slave-8w9v"
server_key: "328be6d90702f2d8f0a7ac0cb94b1ff7"
server_key_iv: "b93496f13954bd9df28d4d33610f373a"
server_key_version: "encryptionkey@0"
I20260807 19:36:16.061582 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:36:16.065218 30286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:16.066541 30270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:36:16.066740 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "a0bb2d180aa145d287a98e080d0bc4f0"
format_stamp: "Formatted at 2026-08-07 19:36:16 on dist-test-slave-8w9v"
server_key: "328be6d90702f2d8f0a7ac0cb94b1ff7"
server_key_iv: "b93496f13954bd9df28d4d33610f373a"
server_key_version: "encryptionkey@0"
I20260807 19:36:16.066962 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:16.094638 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:16.099205 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:16.099546 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:16.100665 30270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:16.102101 30270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:16.102210 30270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:16.102370 30270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:16.102447 30270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:16.137468 30270 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:37067
I20260807 19:36:16.137512 30414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:37067 every 8 connection(s)
I20260807 19:36:16.138866 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:36:16.146081 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30270
I20260807 19:36:16.146281 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:36:16.146762 9058 external_mini_cluster.cc:1428] Setting key 18a1ccf32d28d8f2da8d8626936135dd
Aug 07 19:36:16 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131376, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.131@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:16.161087 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.141616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:36261 (local address 127.8.216.190:33799)
0807 19:36:16.142126 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.142130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.142969 (+ 839us) server_negotiation.cc:407] Connection header received
0807 19:36:16.145022 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.145029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.145118 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.145242 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:16.146983 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.147982 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.150005 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.150512 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.150622 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.154299 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:16.154304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:16.154307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:16.154341 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:16.157460 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.158312 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.158316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.158318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.158449 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.158897 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.158902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.158904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.159288 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:16.159437 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.160270 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.160542 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":280,"threads_started":1}
I20260807 19:36:16.162181 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.142053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:33799 (local address 127.8.216.131:36261)
0807 19:36:16.142785 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.142824 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.144709 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.145479 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.145488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.146062 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:16.146759 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.146778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.148230 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.148234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.149821 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.149840 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.150723 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.150728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.150934 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.151659 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:36:16.151682 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:16.154073 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:16.157609 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:16.157617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:16.157630 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:16.158142 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:16.158545 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:36:16.158548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:36:16.158550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:36:16.158744 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:36:16.159408 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:16.159413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:16.159755 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.160035 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.160612 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":284,"threads_started":1}
I20260807 19:36:16.163776 30415 heartbeater.cc:347] Connected to a master server at 127.8.216.190:33799
I20260807 19:36:16.164077 30415 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:16.164888 30415 heartbeater.cc:511] Master 127.8.216.190:33799 requested a full tablet report, sending...
I20260807 19:36:16.166431 29915 ts_manager.cc:194] Registered new tserver with Master: a0bb2d180aa145d287a98e080d0bc4f0 (127.8.216.131:37067)
I20260807 19:36:16.167382 29915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.216.131@KRBTEST.COM'} at 127.8.216.131:36261
I20260807 19:36:16.176064 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:36:16.194392 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.180572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.8.216.190:33799)
0807 19:36:16.180826 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.180835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.181209 (+ 374us) server_negotiation.cc:407] Connection header received
0807 19:36:16.181456 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.181461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.181547 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.181673 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:16.183170 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.183930 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.185381 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.185840 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.185992 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.188651 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:16.188656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:16.188660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:16.188695 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:16.191430 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.192242 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.192248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.192253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.192394 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.192797 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.192800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.192802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.193145 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:16.193423 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.193802 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.194054 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:36:16.199093 29915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:16.206281 29915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:16.244942 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.219844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:33463 (local address 127.0.0.1:44480)
0807 19:36:16.235022 (+ 15178us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.235038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.235491 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.236341 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.236348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.236383 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:16.236785 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.236794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238671 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.238676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.241650 (+ 2974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.241669 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.243547 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.243554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.243806 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.243882 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.244036 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.244321 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15085,"mutex_wait_us":841,"thread_start_us":13625,"threads_started":1}
I20260807 19:36:16.245057 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.220540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.8.216.129:40397)
0807 19:36:16.231621 (+ 11081us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.231634 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.234722 (+ 3088us) server_negotiation.cc:407] Connection header received
0807 19:36:16.234848 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.234858 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.235185 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.235377 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:16.237187 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238472 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.242458 (+ 3986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.243717 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.243881 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.244184 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.244460 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.244671 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10782,"thread_start_us":10591,"threads_started":1}
I20260807 19:36:16.246023 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.220369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.8.216.130:33463)
0807 19:36:16.233852 (+ 13483us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.233864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.235191 (+ 1327us) server_negotiation.cc:407] Connection header received
0807 19:36:16.235720 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.235731 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.235958 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.236182 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:16.236985 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238542 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.241838 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.243286 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.243458 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.243701 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.245589 (+ 1888us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.245757 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13254,"thread_start_us":13095,"threads_started":1}
I20260807 19:36:16.245198 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.219844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37246)
0807 19:36:16.234387 (+ 14543us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.234412 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.234570 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.235707 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.235716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.235767 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:16.237034 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.237040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238699 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.238705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.242182 (+ 3477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.242202 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.243984 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.243991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.244222 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.244266 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.244354 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.244641 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14403,"mutex_wait_us":893,"spinlock_wait_cycles":7168,"thread_start_us":11853,"threads_started":1}
I20260807 19:36:16.246901 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.219355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37067 (local address 127.0.0.1:59572)
0807 19:36:16.232258 (+ 12903us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.232303 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.232549 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.235707 (+ 3158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.235716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.235766 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:16.236440 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.236456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238641 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.238650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.242314 (+ 3664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.242331 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.245878 (+ 3547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.245885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.246091 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.246137 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.246235 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.246455 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12540,"mutex_wait_us":231,"spinlock_wait_cycles":9856,"thread_start_us":15858,"threads_started":1}
I20260807 19:36:16.247166 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.219769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.8.216.131:37067)
0807 19:36:16.234831 (+ 15062us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.234839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.234867 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:36:16.234951 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.234958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.235165 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.235389 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:16.236660 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.238449 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.242506 (+ 4057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.245625 (+ 3119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.245824 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.246097 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.246499 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.246713 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14877,"thread_start_us":14692,"threads_started":1}
I20260807 19:36:16.251091 30191 tablet_service.cc:1467] Processing CreateTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 (DEFAULT_TABLE table=test-table [id=3d5b4a9969834485bc6f6f9f9e9d3096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:16.251107 30340 tablet_service.cc:1467] Processing CreateTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 (DEFAULT_TABLE table=test-table [id=3d5b4a9969834485bc6f6f9f9e9d3096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:16.251107 30046 tablet_service.cc:1467] Processing CreateTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 (DEFAULT_TABLE table=test-table [id=3d5b4a9969834485bc6f6f9f9e9d3096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:16.252661 30191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28367a1eae4d4252af537a05de3e4aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:16.252780 30340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28367a1eae4d4252af537a05de3e4aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:16.252800 30046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28367a1eae4d4252af537a05de3e4aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:16.267009 30437 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Bootstrap starting.
I20260807 19:36:16.271421 30438 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Bootstrap starting.
I20260807 19:36:16.271837 30439 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Bootstrap starting.
I20260807 19:36:16.271745 30437 tablet_bootstrap.cc:654] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:16.274531 30437 log.cc:824] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:16.276186 30439 tablet_bootstrap.cc:654] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:16.277714 30439 log.cc:824] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:16.277972 30437 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: No bootstrap required, opened a new log
I20260807 19:36:16.278256 30437 ts_tablet_manager.cc:1397] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 19:36:16.279912 30438 tablet_bootstrap.cc:654] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:16.281816 30438 log.cc:824] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:16.285331 30438 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: No bootstrap required, opened a new log
I20260807 19:36:16.285383 30439 tablet_bootstrap.cc:492] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: No bootstrap required, opened a new log
I20260807 19:36:16.285641 30438 ts_tablet_manager.cc:1397] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260807 19:36:16.285676 30439 ts_tablet_manager.cc:1397] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260807 19:36:16.287649 30437 raft_consensus.cc:348] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.288174 30437 raft_consensus.cc:374] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:16.288293 30437 raft_consensus.cc:729] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1dc04ea4374537850a94d9017897b5, State: Initialized, Role: FOLLOWER
I20260807 19:36:16.288998 30437 consensus_queue.cc:260] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.290328 30262 heartbeater.cc:503] Master 127.8.216.190:33799 was elected leader, sending a full tablet report...
I20260807 19:36:16.291070 30437 ts_tablet_manager.cc:1428] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260807 19:36:16.297111 30438 raft_consensus.cc:348] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.297501 30444 raft_consensus.cc:482] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:16.297663 30438 raft_consensus.cc:374] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:16.297821 30444 raft_consensus.cc:504] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.297768 30438 raft_consensus.cc:729] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a682669bdfe472d90526c897c82cc83, State: Initialized, Role: FOLLOWER
I20260807 19:36:16.299314 30438 consensus_queue.cc:260] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.299903 30439 raft_consensus.cc:348] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.300487 30439 raft_consensus.cc:374] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:16.300613 30439 raft_consensus.cc:729] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bb2d180aa145d287a98e080d0bc4f0, State: Initialized, Role: FOLLOWER
I20260807 19:36:16.301287 30439 consensus_queue.cc:260] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.307626 30444 leader_election.cc:288] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bb2d180aa145d287a98e080d0bc4f0 (127.8.216.131:37067), 6a682669bdfe472d90526c897c82cc83 (127.8.216.129:40397)
W20260807 19:36:16.309335 30263 tablet.cc:2369] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:16.310053 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.300511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:52859 (local address 127.8.216.131:37067)
0807 19:36:16.300730 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.300736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.301099 (+ 363us) server_negotiation.cc:407] Connection header received
0807 19:36:16.301458 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.301466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.301585 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.301666 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:16.302771 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.304346 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.307166 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.308500 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.308634 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.308710 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.309623 (+ 913us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.309788 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:36:16.311020 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.300041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:37067 (local address 127.8.216.130:52859)
0807 19:36:16.300969 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.300996 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.301242 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.302013 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.302020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.302057 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:16.302542 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.302557 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.304613 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.304618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.306965 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.306983 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.309015 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.309023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.309229 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.309270 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.310507 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.310664 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":709,"threads_started":1}
I20260807 19:36:16.312602 30362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28367a1eae4d4252af537a05de3e4aa0" candidate_uuid: "8f1dc04ea4374537850a94d9017897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" is_pre_election: true
I20260807 19:36:16.313383 30362 raft_consensus.cc:2454] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1dc04ea4374537850a94d9017897b5 in term 0.
I20260807 19:36:16.314406 30142 leader_election.cc:302] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f1dc04ea4374537850a94d9017897b5, a0bb2d180aa145d287a98e080d0bc4f0; no voters:
I20260807 19:36:16.315037 30444 raft_consensus.cc:2781] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:16.315230 30444 raft_consensus.cc:482] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:16.315358 30444 raft_consensus.cc:3037] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:16.318657 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.302167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:43783 (local address 127.8.216.129:40397)
0807 19:36:16.302425 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.302444 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.311531 (+ 9087us) server_negotiation.cc:407] Connection header received
0807 19:36:16.311795 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.311801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.311875 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.311933 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:16.312489 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.313540 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.316223 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.317464 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.317601 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.317675 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.318236 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.318380 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:36:16.319465 30447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.301576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.8.216.130:43783)
0807 19:36:16.311432 (+ 9856us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.311451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:16.311636 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:16.312113 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:16.312154 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:16.312179 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:16.312327 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.312336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.313828 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.313833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.315993 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:16.316010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.317817 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.317823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:16.318051 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.318084 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:36:16.319009 (+ 925us) client_negotiation.cc:240] Negotiation successful
0807 19:36:16.319176 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9725,"threads_started":1}
I20260807 19:36:16.320986 30067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28367a1eae4d4252af537a05de3e4aa0" candidate_uuid: "8f1dc04ea4374537850a94d9017897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a682669bdfe472d90526c897c82cc83" is_pre_election: true
I20260807 19:36:16.321578 30067 raft_consensus.cc:2454] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1dc04ea4374537850a94d9017897b5 in term 0.
I20260807 19:36:16.321579 30444 raft_consensus.cc:504] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.323266 30444 leader_election.cc:288] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [CANDIDATE]: Term 1 election: Requested vote from peers a0bb2d180aa145d287a98e080d0bc4f0 (127.8.216.131:37067), 6a682669bdfe472d90526c897c82cc83 (127.8.216.129:40397)
I20260807 19:36:16.323298 30362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28367a1eae4d4252af537a05de3e4aa0" candidate_uuid: "8f1dc04ea4374537850a94d9017897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb2d180aa145d287a98e080d0bc4f0"
I20260807 19:36:16.323544 30362 raft_consensus.cc:3037] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:16.324044 30067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28367a1eae4d4252af537a05de3e4aa0" candidate_uuid: "8f1dc04ea4374537850a94d9017897b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a682669bdfe472d90526c897c82cc83"
I20260807 19:36:16.324359 30067 raft_consensus.cc:3037] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:16.327041 30362 raft_consensus.cc:2454] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1dc04ea4374537850a94d9017897b5 in term 1.
I20260807 19:36:16.327694 30142 leader_election.cc:302] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f1dc04ea4374537850a94d9017897b5, a0bb2d180aa145d287a98e080d0bc4f0; no voters:
I20260807 19:36:16.328220 30444 raft_consensus.cc:2781] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:16.328517 30444 raft_consensus.cc:686] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 1 LEADER]: Becoming Leader. State: Replica: 8f1dc04ea4374537850a94d9017897b5, State: Running, Role: LEADER
I20260807 19:36:16.329442 30444 consensus_queue.cc:237] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } }
I20260807 19:36:16.329669 30067 raft_consensus.cc:2454] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1dc04ea4374537850a94d9017897b5 in term 1.
I20260807 19:36:16.332382 30439 ts_tablet_manager.cc:1428] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Time spent starting tablet: real 0.047s user 0.026s sys 0.019s
I20260807 19:36:16.332501 30438 ts_tablet_manager.cc:1428] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Time spent starting tablet: real 0.047s user 0.012s sys 0.011s
I20260807 19:36:16.333105 30415 heartbeater.cc:503] Master 127.8.216.190:33799 was elected leader, sending a full tablet report...
I20260807 19:36:16.336342 29912 catalog_manager.cc:5495] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f1dc04ea4374537850a94d9017897b5 (127.8.216.130). New cstate: current_term: 1 leader_uuid: "8f1dc04ea4374537850a94d9017897b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f1dc04ea4374537850a94d9017897b5" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 33463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 } health_report { overall_health: UNKNOWN } } }
I20260807 19:36:16.371865 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.365521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.8.216.130:33463)
0807 19:36:16.365785 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.365789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.365807 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:36:16.366001 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.366005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.366098 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.366231 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:16.366645 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.367526 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.369489 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.369895 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.370003 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.370618 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:16.371174 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:16.371286 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.371457 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.371539 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:36:16.380702 30362 raft_consensus.cc:1261] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 1 FOLLOWER]: Refusing update from remote peer 8f1dc04ea4374537850a94d9017897b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:16.380704 30067 raft_consensus.cc:1261] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 1 FOLLOWER]: Refusing update from remote peer 8f1dc04ea4374537850a94d9017897b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:16.381769 30443 consensus_queue.cc:1035] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a682669bdfe472d90526c897c82cc83" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:16.382148 30444 consensus_queue.cc:1035] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bb2d180aa145d287a98e080d0bc4f0" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:36:16.390607 30416 tablet.cc:2369] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:16.393698 30452 mvcc.cc:205] Tried to move back new op lower bound from 7315994117644546048 to 7315994117452009472. Current Snapshot: MvccSnapshot[applied={T|T < 7315994117644546048}]
I20260807 19:36:16.398313 30453 mvcc.cc:205] Tried to move back new op lower bound from 7315994117644546048 to 7315994117452009472. Current Snapshot: MvccSnapshot[applied={T|T < 7315994117644546048}]
I20260807 19:36:16.398535 30455 mvcc.cc:205] Tried to move back new op lower bound from 7315994117644546048 to 7315994117452009472. Current Snapshot: MvccSnapshot[applied={T|T < 7315994117644546048}]
I20260807 19:36:16.408344 29915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:16.408675 29915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:16.412671 29915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ca39f87575f458c83407d7970be7019: Sending DeleteTablet for 3 replicas of tablet 28367a1eae4d4252af537a05de3e4aa0
I20260807 19:36:16.413895 30340 tablet_service.cc:1514] Processing DeleteTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:16 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:59572
I20260807 19:36:16.413936 30191 tablet_service.cc:1514] Processing DeleteTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:16 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:44480
I20260807 19:36:16.414674 30046 tablet_service.cc:1514] Processing DeleteTablet for tablet 28367a1eae4d4252af537a05de3e4aa0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:16 UTC) from {username='kudu', principal='kudu/127.8.216.190@KRBTEST.COM'} at 127.0.0.1:37246
I20260807 19:36:16.416114 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29979
I20260807 19:36:16.426134 30462 tablet_replica.cc:330] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: stopping tablet replica
I20260807 19:36:16.427037 30462 raft_consensus.cc:2229] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:16.428336 30462 raft_consensus.cc:2258] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:16.429630 30463 tablet_replica.cc:330] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: stopping tablet replica
I20260807 19:36:16.430312 30463 raft_consensus.cc:2229] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:16.430975 30463 raft_consensus.cc:2258] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:16.431655 30462 ts_tablet_manager.cc:1905] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:16.436729 30463 ts_tablet_manager.cc:1905] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:16.438723 30464 tablet_replica.cc:330] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: stopping tablet replica
Aug 07 19:36:16 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.129@KRBTEST.COM
I20260807 19:36:16.439211 30464 raft_consensus.cc:2229] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:16.439848 30464 raft_consensus.cc:2258] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:16.441696 30462 ts_tablet_manager.cc:1918] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:16.441923 30462 log.cc:1196] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/28367a1eae4d4252af537a05de3e4aa0
I20260807 19:36:16.442579 30462 ts_tablet_manager.cc:1939] T 28367a1eae4d4252af537a05de3e4aa0 P 8f1dc04ea4374537850a94d9017897b5: Deleting consensus metadata
I20260807 19:36:16.444785 29899 catalog_manager.cc:4841] TS 8f1dc04ea4374537850a94d9017897b5 (127.8.216.130:33463): tablet 28367a1eae4d4252af537a05de3e4aa0 (table test-table [id=3d5b4a9969834485bc6f6f9f9e9d3096]) successfully deleted
I20260807 19:36:16.446020 30463 ts_tablet_manager.cc:1918] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:16.446218 30463 log.cc:1196] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/28367a1eae4d4252af537a05de3e4aa0
I20260807 19:36:16.446570 30464 ts_tablet_manager.cc:1905] T 28367a1eae4d4252af537a05de3e4aa0 P 6a682669bdfe472d90526c897c82cc83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:16.446853 30463 ts_tablet_manager.cc:1939] T 28367a1eae4d4252af537a05de3e4aa0 P a0bb2d180aa145d287a98e080d0bc4f0: Deleting consensus metadata
I20260807 19:36:16.448988 29900 catalog_manager.cc:4841] TS a0bb2d180aa145d287a98e080d0bc4f0 (127.8.216.131:37067): tablet 28367a1eae4d4252af537a05de3e4aa0 (table test-table [id=3d5b4a9969834485bc6f6f9f9e9d3096]) successfully deleted
I20260807 19:36:16.450184 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.417803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.8.216.129:40397)
0807 19:36:16.418034 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.418044 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.423135 (+ 5091us) server_negotiation.cc:407] Connection header received
0807 19:36:16.424072 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.424078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.424200 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.424283 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:16.428008 (+ 3725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.429254 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.432009 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.433090 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.433260 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.440358 (+ 7098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:16.440376 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:16.440393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:16.440465 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:16.445506 (+ 5041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.446724 (+ 1218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.446735 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.446739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.446980 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.447486 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.447492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.447495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.448036 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:16.448658 (+ 622us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.449462 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.449732 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:36:16.452108 30111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:16.552491 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29979
I20260807 19:36:16.576273 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30123
W20260807 19:36:16.576148 29901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.216.129:40397 (error 108)
Aug 07 19:36:16 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.130@KRBTEST.COM
I20260807 19:36:16.592065 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.591418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37256)
0807 19:36:16.591739 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.591870 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260807 19:36:16.592413 29901 catalog_manager.cc:4584] TS 6a682669bdfe472d90526c897c82cc83 (127.8.216.129:40397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28367a1eae4d4252af537a05de3e4aa0: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
I20260807 19:36:16.599292 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.578200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.8.216.130:33463)
0807 19:36:16.578465 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.578475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.578499 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:36:16.578584 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.578592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.578697 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.578822 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:16.581353 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.582996 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.585683 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.586660 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.586824 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.592521 (+ 5697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:16.592542 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:16.592565 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:16.592616 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:16.595977 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.596917 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.596925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.596927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.597099 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.597594 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.597598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.597600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.597995 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:16.598334 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.598662 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.598924 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 19:36:16.600019 30256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:16.646158 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.645556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37264)
0807 19:36:16.645830 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.645945 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 19:36:16.647369 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.646943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37268)
0807 19:36:16.647123 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.647225 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:36:16.708143 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.707488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37270)
0807 19:36:16.707777 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.707893 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 19:36:16.709225 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.708820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37278)
0807 19:36:16.708995 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.709083 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:36:16.721589 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30123
I20260807 19:36:16.745966 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30270
Aug 07 19:36:16 dist-test-slave-8w9v krb5kdc[29867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.131@KRBTEST.COM
I20260807 19:36:16.761776 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.746821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.8.216.131:37067)
0807 19:36:16.746996 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:16.747001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:16.747038 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:36:16.747207 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:16.747211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:16.747277 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:16.747363 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:16.748818 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.749632 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.751034 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:16.751416 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:16.751508 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:16.755956 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:16.755982 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:16.755999 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:16.756036 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:16.758928 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.759588 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.759595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.759597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.759723 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:16.760045 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:16.760048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:16.760050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:16.760642 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:16.760859 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:16.761238 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 19:36:16.761444 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:36:16.762447 30408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:16.789933 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.789434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37284)
0807 19:36:16.789682 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.789773 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:36:16.791023 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:16.790617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:40397 (local address 127.0.0.1:37292)
0807 19:36:16.790827 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:16.790907 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.129:40397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 19:36:16.855341 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30270
I20260807 19:36:16.876298 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 29881
I20260807 19:36:16.877502 29946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:16.993402 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 29881
2026-08-07T19:36:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4693 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T19:36:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:17Z Disabled control of system clock
I20260807 19:36:17.093533 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45455
--webserver_interface=127.8.216.190
--webserver_port=0
--builtin_ntp_servers=127.8.216.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:17.469952 30476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:17.470335 30476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:17.470498 30476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:17.481287 30476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:17.481463 30476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:17.481556 30476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:17.481650 30476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:17.481720 30476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:17.492363 30476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:17.493808 30476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:17.495752 30476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:17.508472 30482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:17.508633 30484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:17.508605 30481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:17.509728 30476 server_base.cc:1034] running on GCE node
I20260807 19:36:17.510596 30476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:17.512247 30476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:17.513494 30476 hybrid_clock.cc:648] HybridClock initialized: now 1786131377513444 us; error 63 us; skew 500 ppm
I20260807 19:36:17.513867 30476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:17.516414 30476 webserver.cc:460] Webserver started at http://127.8.216.190:46851/ using document root <none> and password file <none>
I20260807 19:36:17.517151 30476 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:17.517290 30476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:17.517642 30476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:17.520815 30476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "210f002df31f4ecda9cfc36061c63105"
format_stamp: "Formatted at 2026-08-07 19:36:17 on dist-test-slave-8w9v"
server_key: "f554ff1d9ff4c32fc2c5669a77f80988"
server_key_iv: "5f7910ab52c2300fc3b32e58e98b28d7"
server_key_version: "encryptionkey@0"
I20260807 19:36:17.521675 30476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "210f002df31f4ecda9cfc36061c63105"
format_stamp: "Formatted at 2026-08-07 19:36:17 on dist-test-slave-8w9v"
server_key: "f554ff1d9ff4c32fc2c5669a77f80988"
server_key_iv: "5f7910ab52c2300fc3b32e58e98b28d7"
server_key_version: "encryptionkey@0"
I20260807 19:36:17.527169 30476 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 19:36:17.531111 30491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:17.532657 30476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:17.532869 30476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "210f002df31f4ecda9cfc36061c63105"
format_stamp: "Formatted at 2026-08-07 19:36:17 on dist-test-slave-8w9v"
server_key: "f554ff1d9ff4c32fc2c5669a77f80988"
server_key_iv: "5f7910ab52c2300fc3b32e58e98b28d7"
server_key_version: "encryptionkey@0"
I20260807 19:36:17.533120 30476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:17.553920 30476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:17.554858 30476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:17.555176 30476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:17.575063 30476 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:45455
I20260807 19:36:17.575099 30546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:45455 every 8 connection(s)
I20260807 19:36:17.576604 30476 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:17.580840 30547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:17.586460 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30476
I20260807 19:36:17.586683 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:17.587309 9058 external_mini_cluster.cc:1428] Setting key df7ed537b5dee905e8ef4cb05dd223a2
I20260807 19:36:17.594056 30547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: Bootstrap starting.
I20260807 19:36:17.597702 30547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:17.599046 30547 log.cc:824] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:17.602083 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:17.589622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.8.216.190:45455)
0807 19:36:17.590751 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:17.590762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:17.590807 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:36:17.592800 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:17.592827 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:17.593325 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:17.593740 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:17.594334 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:17.595647 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:17.597149 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:17.597913 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:17.598137 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:17.599423 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:17.599432 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:17.599445 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:17.600289 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:17.600409 (+ 120us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:17.600440 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:17.600565 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:17.600827 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 19:36:17.601152 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":424,"threads_started":1}
I20260807 19:36:17.602591 30547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: No bootstrap required, opened a new log
I20260807 19:36:17.610577 30547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } }
I20260807 19:36:17.611054 30547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:17.611202 30547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210f002df31f4ecda9cfc36061c63105, State: Initialized, Role: FOLLOWER
I20260807 19:36:17.611881 30547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } }
I20260807 19:36:17.612179 30547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:17.612370 30547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:17.612561 30547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:17.615008 30547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } }
I20260807 19:36:17.615602 30547 leader_election.cc:302] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210f002df31f4ecda9cfc36061c63105; no voters:
I20260807 19:36:17.616330 30547 leader_election.cc:288] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:17.616457 30552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:17.617267 30552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [term 1 LEADER]: Becoming Leader. State: Replica: 210f002df31f4ecda9cfc36061c63105, State: Running, Role: LEADER
I20260807 19:36:17.617895 30552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } }
I20260807 19:36:17.619907 30547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:17.621042 30554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210f002df31f4ecda9cfc36061c63105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } } }
I20260807 19:36:17.621038 30553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210f002df31f4ecda9cfc36061c63105. Latest consensus state: current_term: 1 leader_uuid: "210f002df31f4ecda9cfc36061c63105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f002df31f4ecda9cfc36061c63105" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45455 } } }
I20260807 19:36:17.621496 30554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:17.621507 30553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:17.624701 30557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:17.629103 30557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:17.639025 30557 catalog_manager.cc:1347] Generated new cluster ID: e45cf5b3349943a18bb7932c9268d8a0
I20260807 19:36:17.639189 30557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:17.666109 30557 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:17.668154 30557 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:17.682997 30557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: Generated new TSK 0
I20260807 19:36:17.684222 30557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:17.690289 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.129:0
--local_ip_for_outbound_sockets=127.8.216.129
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--builtin_ntp_servers=127.8.216.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:36:18.054406 30573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:18.054757 30573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:18.054929 30573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:18.062459 30573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:18.062615 30573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:18.062793 30573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.129
I20260807 19:36:18.072854 30573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.216.129
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.129
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:18.074445 30573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:18.076455 30573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:18.090783 30584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:18.090735 30579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:18.090834 30581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:18.092048 30573 server_base.cc:1034] running on GCE node
I20260807 19:36:18.092784 30573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:18.093905 30573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:18.095175 30573 hybrid_clock.cc:648] HybridClock initialized: now 1786131378095104 us; error 62 us; skew 500 ppm
I20260807 19:36:18.095575 30573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:18.098165 30573 webserver.cc:460] Webserver started at http://127.8.216.129:46693/ using document root <none> and password file <none>
I20260807 19:36:18.098927 30573 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:18.099069 30573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:18.099419 30573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:18.102566 30573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data/instance:
uuid: "91549ed888834d2cb5144b6cc0d3de70"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "a664c1e8c16e044b723e799bd663ec6a"
server_key_iv: "4ad4b6bac9104c4df5d4d5a182407632"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.103586 30573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance:
uuid: "91549ed888834d2cb5144b6cc0d3de70"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "a664c1e8c16e044b723e799bd663ec6a"
server_key_iv: "4ad4b6bac9104c4df5d4d5a182407632"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.109191 30573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:36:18.112962 30590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:18.114423 30573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:18.114617 30573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal
uuid: "91549ed888834d2cb5144b6cc0d3de70"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "a664c1e8c16e044b723e799bd663ec6a"
server_key_iv: "4ad4b6bac9104c4df5d4d5a182407632"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.114867 30573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:18.129371 30573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:18.130311 30573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:18.130671 30573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:18.131841 30573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:18.133546 30573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:18.133685 30573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:18.133816 30573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:18.133905 30573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:18.170049 30573 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.129:42981
I20260807 19:36:18.170105 30708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.129:42981 every 8 connection(s)
I20260807 19:36:18.171603 30573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/data/info.pb
I20260807 19:36:18.175911 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30573
I20260807 19:36:18.176110 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal/instance
I20260807 19:36:18.176733 9058 external_mini_cluster.cc:1428] Setting key 8c4eebc2eb442e61581453b1fc49c640
I20260807 19:36:18.179837 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.130:0
--local_ip_for_outbound_sockets=127.8.216.130
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--builtin_ntp_servers=127.8.216.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:18.185833 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:18.174474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.129:34309 (local address 127.8.216.190:45455)
0807 19:36:18.174999 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:18.175007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:18.176525 (+ 1518us) server_negotiation.cc:407] Connection header received
0807 19:36:18.178653 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:18.178659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:18.178757 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:18.178899 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:18.180109 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.180911 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.182720 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.183326 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.183469 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:18.184946 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:18.184955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:18.184958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:18.184980 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:18.185019 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:18.185032 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:18.185131 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:18.185366 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 19:36:18.185473 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":238,"threads_started":1}
I20260807 19:36:18.186960 30712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:18.175092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45455 (local address 127.8.216.129:34309)
0807 19:36:18.176323 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:18.176376 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:18.178354 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:18.179128 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:18.179138 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:18.179232 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:18.179850 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:18.179868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.181079 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.181090 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:18.182563 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:18.182577 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.183572 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.183580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:18.183818 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:18.184602 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:18.184605 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:18.184649 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:18.184733 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:18.185131 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:18.185146 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:18.185157 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:36:18.185353 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 19:36:18.185802 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":353,"threads_started":1}
I20260807 19:36:18.188654 30709 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45455
I20260807 19:36:18.188988 30709 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:18.189780 30709 heartbeater.cc:511] Master 127.8.216.190:45455 requested a full tablet report, sending...
I20260807 19:36:18.191934 30510 ts_manager.cc:194] Registered new tserver with Master: 91549ed888834d2cb5144b6cc0d3de70 (127.8.216.129:42981)
I20260807 19:36:18.194017 30510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.129:34309
W20260807 19:36:18.539146 30713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:18.539446 30713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:18.539628 30713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:18.547179 30713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:18.547292 30713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:18.547472 30713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.130
I20260807 19:36:18.557299 30713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.216.130
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.130
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:18.558632 30713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:18.560541 30713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:18.573922 30719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:18.573956 30720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:18.575035 30713 server_base.cc:1034] running on GCE node
W20260807 19:36:18.573922 30723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:18.576867 30713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:18.577885 30713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:18.579142 30713 hybrid_clock.cc:648] HybridClock initialized: now 1786131378579086 us; error 48 us; skew 500 ppm
I20260807 19:36:18.579506 30713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:18.582049 30713 webserver.cc:460] Webserver started at http://127.8.216.130:36645/ using document root <none> and password file <none>
I20260807 19:36:18.582788 30713 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:18.582923 30713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:18.583256 30713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:18.586318 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data/instance:
uuid: "14d8fa8c50ad4db8bb6d711f205d3215"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "9fe4b3a345ffa34bc0f8d284131f0aa9"
server_key_iv: "6525dc3261a11e9791dc98bb41b218db"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.587141 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance:
uuid: "14d8fa8c50ad4db8bb6d711f205d3215"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "9fe4b3a345ffa34bc0f8d284131f0aa9"
server_key_iv: "6525dc3261a11e9791dc98bb41b218db"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.592700 30713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:36:18.596597 30731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:18.597998 30713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:18.598194 30713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal
uuid: "14d8fa8c50ad4db8bb6d711f205d3215"
format_stamp: "Formatted at 2026-08-07 19:36:18 on dist-test-slave-8w9v"
server_key: "9fe4b3a345ffa34bc0f8d284131f0aa9"
server_key_iv: "6525dc3261a11e9791dc98bb41b218db"
server_key_version: "encryptionkey@0"
I20260807 19:36:18.598435 30713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:18.615116 30713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:18.616027 30713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:18.616384 30713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:18.617440 30713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:18.619140 30713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:18.619266 30713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:18.619402 30713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:18.619573 30713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 19:36:18.654680 30713 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.130:38303
I20260807 19:36:18.654767 30851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.130:38303 every 8 connection(s)
I20260807 19:36:18.656140 30713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/data/info.pb
I20260807 19:36:18.665453 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30713
I20260807 19:36:18.665648 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal/instance
I20260807 19:36:18.666103 9058 external_mini_cluster.cc:1428] Setting key b5ce99896fd58961ead2f8ae39352083
I20260807 19:36:18.669044 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:18.658993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:50903 (local address 127.8.216.190:45455)
0807 19:36:18.659185 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:18.659189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:18.660614 (+ 1425us) server_negotiation.cc:407] Connection header received
0807 19:36:18.662439 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:18.662445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:18.662542 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:18.662660 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:18.663724 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.664539 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.666198 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.666618 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.666758 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:18.668222 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:18.668228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:18.668235 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:18.668257 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:18.668290 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:18.668302 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:18.668396 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:18.668625 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 19:36:18.668721 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:36:18.669140 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.216.131:0
--local_ip_for_outbound_sockets=127.8.216.131
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--builtin_ntp_servers=127.8.216.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:36:18.670069 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:18.659471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45455 (local address 127.8.216.130:50903)
0807 19:36:18.660456 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:18.660499 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:18.662172 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:18.662893 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:18.662901 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:18.662987 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:18.663559 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:18.663576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.664688 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.664698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:18.666051 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:18.666063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:18.666850 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:18.666857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:18.667087 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:18.667899 (+ 812us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:18.667902 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:18.667945 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:18.668020 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:18.668411 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:18.668435 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:18.668441 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:36:18.668617 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 19:36:18.669017 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":289,"threads_started":1}
I20260807 19:36:18.671494 30852 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45455
I20260807 19:36:18.671813 30852 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:18.672652 30852 heartbeater.cc:511] Master 127.8.216.190:45455 requested a full tablet report, sending...
I20260807 19:36:18.674264 30510 ts_manager.cc:194] Registered new tserver with Master: 14d8fa8c50ad4db8bb6d711f205d3215 (127.8.216.130:38303)
I20260807 19:36:18.675477 30510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.130:50903
W20260807 19:36:19.024893 30855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:19.025203 30855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:19.025297 30855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:19.033013 30855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:19.033178 30855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:19.033434 30855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.216.131
I20260807 19:36:19.043392 30855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.216.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.216.131
--webserver_port=0
--tserver_master_addrs=127.8.216.190:45455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.216.131
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:19.044840 30855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:19.046661 30855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:19.060711 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:19.060954 30864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:19.060676 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:19.061853 30855 server_base.cc:1034] running on GCE node
I20260807 19:36:19.062530 30855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:19.063493 30855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:19.064693 30855 hybrid_clock.cc:648] HybridClock initialized: now 1786131379064648 us; error 35 us; skew 500 ppm
I20260807 19:36:19.065038 30855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:36:19.067548 30855 webserver.cc:460] Webserver started at http://127.8.216.131:38459/ using document root <none> and password file <none>
I20260807 19:36:19.068385 30855 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:19.068522 30855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:19.068861 30855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:19.071830 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data/instance:
uuid: "7c288f5a8ff84db3bbfd1c32f2192db1"
format_stamp: "Formatted at 2026-08-07 19:36:19 on dist-test-slave-8w9v"
server_key: "5898cb82bc52a9edfb475e8c83d5472a"
server_key_iv: "0e21e42d94512193c3d93823512374ef"
server_key_version: "encryptionkey@0"
I20260807 19:36:19.072722 30855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance:
uuid: "7c288f5a8ff84db3bbfd1c32f2192db1"
format_stamp: "Formatted at 2026-08-07 19:36:19 on dist-test-slave-8w9v"
server_key: "5898cb82bc52a9edfb475e8c83d5472a"
server_key_iv: "0e21e42d94512193c3d93823512374ef"
server_key_version: "encryptionkey@0"
I20260807 19:36:19.078105 30855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:36:19.082064 30871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:19.083544 30855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:36:19.083734 30855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal
uuid: "7c288f5a8ff84db3bbfd1c32f2192db1"
format_stamp: "Formatted at 2026-08-07 19:36:19 on dist-test-slave-8w9v"
server_key: "5898cb82bc52a9edfb475e8c83d5472a"
server_key_iv: "0e21e42d94512193c3d93823512374ef"
server_key_version: "encryptionkey@0"
I20260807 19:36:19.083997 30855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:19.097885 30855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:19.098771 30855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:19.099100 30855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:19.100112 30855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:36:19.101717 30855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:36:19.101809 30855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:19.101958 30855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:36:19.102022 30855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:19.137651 30855 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.131:46753
I20260807 19:36:19.137715 30992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.131:46753 every 8 connection(s)
I20260807 19:36:19.139168 30855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/data/info.pb
I20260807 19:36:19.143959 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 30855
I20260807 19:36:19.144133 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal/instance
I20260807 19:36:19.144697 9058 external_mini_cluster.cc:1428] Setting key 72b2e1a8967883c7d16d74a6a9ff6d00
I20260807 19:36:19.152895 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.141883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.131:45331 (local address 127.8.216.190:45455)
0807 19:36:19.142072 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.142077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.143521 (+ 1444us) server_negotiation.cc:407] Connection header received
0807 19:36:19.145925 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.145929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.146023 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.146139 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:19.147576 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.148586 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.150166 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.150580 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.150673 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.152100 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:19.152104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:19.152107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:19.152174 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:19.152214 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:19.152228 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:19.152297 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.152566 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.152647 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:36:19.153997 30995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.142428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45455 (local address 127.8.216.131:45331)
0807 19:36:19.143352 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.143389 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.145630 (+ 2241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.146450 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.146464 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.146595 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:36:19.147398 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.147417 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.148740 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.148745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.150040 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.150053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.150792 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.150797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.151003 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.151801 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:36:19.151804 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:36:19.151848 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:36:19.151930 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:36:19.152353 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:36:19.152363 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:36:19.152370 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.152535 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.152868 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":275,"threads_started":1}
I20260807 19:36:19.155547 30993 heartbeater.cc:347] Connected to a master server at 127.8.216.190:45455
I20260807 19:36:19.155831 30993 heartbeater.cc:464] Registering TS with master...
I20260807 19:36:19.156663 30993 heartbeater.cc:511] Master 127.8.216.190:45455 requested a full tablet report, sending...
I20260807 19:36:19.158121 30510 ts_manager.cc:194] Registered new tserver with Master: 7c288f5a8ff84db3bbfd1c32f2192db1 (127.8.216.131:46753)
I20260807 19:36:19.159071 30510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.216.131:45331
I20260807 19:36:19.160881 9058 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:36:19.171952 30711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.165258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.8.216.190:45455)
0807 19:36:19.165434 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.165439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.165935 (+ 496us) server_negotiation.cc:407] Connection header received
0807 19:36:19.166086 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.166090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.166169 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.166266 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:19.166747 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.167468 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.169161 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.169857 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.169994 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.171207 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:19.171213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:19.171218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:19.171241 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:19.171284 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:19.171300 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:19.171388 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.171517 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.171616 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:36:19.176189 30510 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:36:19.183383 30510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:36:19.197615 30709 heartbeater.cc:503] Master 127.8.216.190:45455 was elected leader, sending a full tablet report...
I20260807 19:36:19.222430 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.198339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.8.216.131:46753)
0807 19:36:19.199335 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.199348 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.210442 (+ 11094us) server_negotiation.cc:407] Connection header received
0807 19:36:19.210818 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.210830 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.211191 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.211414 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:19.212660 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.216162 (+ 3502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.219328 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.220870 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.221010 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.221343 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.221870 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.222050 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":534,"threads_started":1}
I20260807 19:36:19.222494 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.196849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:46753 (local address 127.0.0.1:38478)
0807 19:36:19.210245 (+ 13396us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.210274 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.210587 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.211895 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.211903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.211944 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:19.212382 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.212396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.216386 (+ 3990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.216392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.219118 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.219135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.221479 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.221484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.221628 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.221662 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.221819 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.222023 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13192,"mutex_wait_us":139,"thread_start_us":13322,"threads_started":1}
I20260807 19:36:19.223176 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.196989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.8.216.129:42981)
0807 19:36:19.212457 (+ 15468us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.212469 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.212506 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:36:19.212625 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.212636 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.212911 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.214063 (+ 1152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:19.214370 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.215999 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.219171 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.220870 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.221010 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.221361 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.222607 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.222785 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15238,"thread_start_us":15046,"threads_started":1}
I20260807 19:36:19.223968 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.196621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:42981 (local address 127.0.0.1:46242)
0807 19:36:19.209645 (+ 13024us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.209723 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.209964 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.213186 (+ 3222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.213194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.213228 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:19.213807 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.213823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.216165 (+ 2342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.216173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.218976 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.218987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.222169 (+ 3182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.222175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.222373 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.222408 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.223465 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.223605 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12635,"mutex_wait_us":190,"thread_start_us":13221,"threads_started":1}
I20260807 19:36:19.225395 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.212447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.130:38303 (local address 127.0.0.1:57950)
0807 19:36:19.216748 (+ 4301us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.216773 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.216912 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.217596 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.217605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.217640 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:19.218159 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.218178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.220103 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.220108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.221869 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.221882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.224577 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.224584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.224756 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.224799 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.224903 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.225037 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4157,"mutex_wait_us":57,"thread_start_us":3912,"threads_started":1}
I20260807 19:36:19.225668 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.211014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.8.216.130:38303)
0807 19:36:19.215715 (+ 4701us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.215724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.216911 (+ 1187us) server_negotiation.cc:407] Connection header received
0807 19:36:19.217111 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.217120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.217380 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.217583 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:19.218370 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.219922 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.222077 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.223981 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.224173 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.224448 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.224983 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.225152 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4546,"thread_start_us":4422,"threads_started":1}
I20260807 19:36:19.226553 30923 tablet_service.cc:1467] Processing CreateTablet for tablet 526e77e34c024571adfe16304a5167e4 (DEFAULT_TABLE table=test-table [id=6e83b0b103e74df28007b8457e2c4347]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:19.227996 30923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526e77e34c024571adfe16304a5167e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:19.227566 30641 tablet_service.cc:1467] Processing CreateTablet for tablet 526e77e34c024571adfe16304a5167e4 (DEFAULT_TABLE table=test-table [id=6e83b0b103e74df28007b8457e2c4347]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:19.229059 30641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526e77e34c024571adfe16304a5167e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:19.231135 30783 tablet_service.cc:1467] Processing CreateTablet for tablet 526e77e34c024571adfe16304a5167e4 (DEFAULT_TABLE table=test-table [id=6e83b0b103e74df28007b8457e2c4347]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:36:19.232713 30783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 526e77e34c024571adfe16304a5167e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:19.240360 31011 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Bootstrap starting.
I20260807 19:36:19.243681 31011 tablet_bootstrap.cc:654] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:19.244697 31012 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Bootstrap starting.
I20260807 19:36:19.245157 31011 log.cc:824] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:19.248433 31014 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Bootstrap starting.
I20260807 19:36:19.248624 31012 tablet_bootstrap.cc:654] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:19.250073 31012 log.cc:824] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:19.250401 31011 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: No bootstrap required, opened a new log
I20260807 19:36:19.250735 31011 ts_tablet_manager.cc:1397] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260807 19:36:19.251981 31014 tablet_bootstrap.cc:654] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:19.253054 31014 log.cc:824] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:19.253575 31012 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: No bootstrap required, opened a new log
I20260807 19:36:19.253865 31012 ts_tablet_manager.cc:1397] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:36:19.256438 31014 tablet_bootstrap.cc:492] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: No bootstrap required, opened a new log
I20260807 19:36:19.256752 31014 ts_tablet_manager.cc:1397] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:36:19.262562 31011 raft_consensus.cc:348] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.263154 31011 raft_consensus.cc:374] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:19.263266 31011 raft_consensus.cc:729] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c288f5a8ff84db3bbfd1c32f2192db1, State: Initialized, Role: FOLLOWER
I20260807 19:36:19.264008 31011 consensus_queue.cc:260] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.265322 30993 heartbeater.cc:503] Master 127.8.216.190:45455 was elected leader, sending a full tablet report...
I20260807 19:36:19.265630 31014 raft_consensus.cc:348] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.266052 31014 raft_consensus.cc:374] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:19.266083 31011 ts_tablet_manager.cc:1428] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260807 19:36:19.266178 31014 raft_consensus.cc:729] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d8fa8c50ad4db8bb6d711f205d3215, State: Initialized, Role: FOLLOWER
I20260807 19:36:19.266386 31012 raft_consensus.cc:348] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.266901 31012 raft_consensus.cc:374] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:19.267040 31012 raft_consensus.cc:729] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91549ed888834d2cb5144b6cc0d3de70, State: Initialized, Role: FOLLOWER
I20260807 19:36:19.266856 31014 consensus_queue.cc:260] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.267700 31012 consensus_queue.cc:260] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.268163 30852 heartbeater.cc:503] Master 127.8.216.190:45455 was elected leader, sending a full tablet report...
I20260807 19:36:19.268932 31014 ts_tablet_manager.cc:1428] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 19:36:19.271399 31012 ts_tablet_manager.cc:1428] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
W20260807 19:36:19.391033 30994 tablet.cc:2369] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:19.408519 30853 tablet.cc:2369] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:36:19.424299 30710 tablet.cc:2369] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:36:19.636439 31019 raft_consensus.cc:482] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:36:19.636816 31019 raft_consensus.cc:504] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.638616 31019 leader_election.cc:288] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91549ed888834d2cb5144b6cc0d3de70 (127.8.216.129:42981), 7c288f5a8ff84db3bbfd1c32f2192db1 (127.8.216.131:46753)
I20260807 19:36:19.648082 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.639169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:56613 (local address 127.8.216.131:46753)
0807 19:36:19.639497 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.639504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.639938 (+ 434us) server_negotiation.cc:407] Connection header received
0807 19:36:19.640179 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.640184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.640259 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.640364 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:19.641298 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.642460 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.645691 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.646939 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.647127 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.647197 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.647677 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.647810 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260807 19:36:19.648082 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.639139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.216.130:46519 (local address 127.8.216.129:42981)
0807 19:36:19.639383 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.639392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.640034 (+ 642us) server_negotiation.cc:407] Connection header received
0807 19:36:19.640167 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.640172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.640269 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.640364 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:36:19.641298 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.642407 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.645688 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.646961 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.647123 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.647197 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.647677 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.647810 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:36:19.648310 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.639066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.129:42981 (local address 127.8.216.130:46519)
0807 19:36:19.639919 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.639935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.640031 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.640399 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.640405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.640433 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:19.641090 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.641104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.642599 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.642603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.645429 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.645446 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.647205 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.647212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.647465 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.647526 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.647624 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.647820 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":202,"thread_start_us":531,"threads_started":1}
I20260807 19:36:19.648334 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.638946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.131:46753 (local address 127.8.216.130:56613)
0807 19:36:19.639811 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0807 19:36:19.639833 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:36:19.640009 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:36:19.640421 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:36:19.640425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:36:19.640441 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:36:19.641093 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.641104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.642598 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.642603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.645438 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:36:19.645446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.647200 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.647206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:36:19.647470 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.647522 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:36:19.647624 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:36:19.647820 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":335,"thread_start_us":438,"threads_started":1}
I20260807 19:36:19.649778 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526e77e34c024571adfe16304a5167e4" candidate_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" is_pre_election: true
I20260807 19:36:19.649778 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526e77e34c024571adfe16304a5167e4" candidate_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91549ed888834d2cb5144b6cc0d3de70" is_pre_election: true
I20260807 19:36:19.650415 30946 raft_consensus.cc:2454] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d8fa8c50ad4db8bb6d711f205d3215 in term 0.
I20260807 19:36:19.650416 30661 raft_consensus.cc:2454] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d8fa8c50ad4db8bb6d711f205d3215 in term 0.
I20260807 19:36:19.651234 30732 leader_election.cc:302] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14d8fa8c50ad4db8bb6d711f205d3215, 7c288f5a8ff84db3bbfd1c32f2192db1, 91549ed888834d2cb5144b6cc0d3de70; no voters:
I20260807 19:36:19.651762 31019 raft_consensus.cc:2781] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:36:19.651933 31019 raft_consensus.cc:482] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:36:19.652050 31019 raft_consensus.cc:3037] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:19.654675 31019 raft_consensus.cc:504] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.655439 31019 leader_election.cc:288] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [CANDIDATE]: Term 1 election: Requested vote from peers 91549ed888834d2cb5144b6cc0d3de70 (127.8.216.129:42981), 7c288f5a8ff84db3bbfd1c32f2192db1 (127.8.216.131:46753)
I20260807 19:36:19.656002 30661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526e77e34c024571adfe16304a5167e4" candidate_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91549ed888834d2cb5144b6cc0d3de70"
I20260807 19:36:19.656201 30946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "526e77e34c024571adfe16304a5167e4" candidate_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1"
I20260807 19:36:19.656278 30661 raft_consensus.cc:3037] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:19.656446 30946 raft_consensus.cc:3037] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:19.659715 30946 raft_consensus.cc:2454] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d8fa8c50ad4db8bb6d711f205d3215 in term 1.
I20260807 19:36:19.659715 30661 raft_consensus.cc:2454] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d8fa8c50ad4db8bb6d711f205d3215 in term 1.
I20260807 19:36:19.660208 30732 leader_election.cc:302] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d8fa8c50ad4db8bb6d711f205d3215, 91549ed888834d2cb5144b6cc0d3de70; no voters:
I20260807 19:36:19.660588 31019 raft_consensus.cc:2781] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:19.661180 31019 raft_consensus.cc:686] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 1 LEADER]: Becoming Leader. State: Replica: 14d8fa8c50ad4db8bb6d711f205d3215, State: Running, Role: LEADER
I20260807 19:36:19.661746 31019 consensus_queue.cc:237] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } }
I20260807 19:36:19.667177 30510 catalog_manager.cc:5495] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14d8fa8c50ad4db8bb6d711f205d3215 (127.8.216.130). New cstate: current_term: 1 leader_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14d8fa8c50ad4db8bb6d711f205d3215" member_type: VOTER last_known_addr { host: "127.8.216.130" port: 38303 } health_report { overall_health: HEALTHY } } }
I20260807 19:36:19.777187 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.770680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.8.216.130:38303)
0807 19:36:19.771523 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.771531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.771557 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:36:19.771673 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.771677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.771798 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.771983 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:36:19.772525 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.773241 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.774744 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.775155 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.775315 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.775848 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:36:19.776510 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:36:19.776683 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.776768 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.776859 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":589,"threads_started":1}
I20260807 19:36:19.785835 30946 raft_consensus.cc:1261] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 1 FOLLOWER]: Refusing update from remote peer 14d8fa8c50ad4db8bb6d711f205d3215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:19.785830 30661 raft_consensus.cc:1261] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 1 FOLLOWER]: Refusing update from remote peer 14d8fa8c50ad4db8bb6d711f205d3215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:36:19.786886 31023 consensus_queue.cc:1035] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c288f5a8ff84db3bbfd1c32f2192db1" member_type: VOTER last_known_addr { host: "127.8.216.131" port: 46753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:19.787290 31019 consensus_queue.cc:1035] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91549ed888834d2cb5144b6cc0d3de70" member_type: VOTER last_known_addr { host: "127.8.216.129" port: 42981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:36:19.822240 30509 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:19.822566 30509 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:36:19.826572 30509 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210f002df31f4ecda9cfc36061c63105: Sending DeleteTablet for 3 replicas of tablet 526e77e34c024571adfe16304a5167e4
I20260807 19:36:19.827802 30923 tablet_service.cc:1514] Processing DeleteTablet for tablet 526e77e34c024571adfe16304a5167e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:19 UTC) from {username='slave'} at 127.0.0.1:38478
I20260807 19:36:19.827852 30641 tablet_service.cc:1514] Processing DeleteTablet for tablet 526e77e34c024571adfe16304a5167e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:19 UTC) from {username='slave'} at 127.0.0.1:46242
I20260807 19:36:19.828306 30783 tablet_service.cc:1514] Processing DeleteTablet for tablet 526e77e34c024571adfe16304a5167e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:36:19 UTC) from {username='slave'} at 127.0.0.1:57950
I20260807 19:36:19.829025 31044 tablet_replica.cc:330] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: stopping tablet replica
I20260807 19:36:19.829658 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30573
I20260807 19:36:19.829766 31044 raft_consensus.cc:2229] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:19.830293 31044 raft_consensus.cc:2258] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:19.832469 31044 ts_tablet_manager.cc:1905] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:19.841641 31044 ts_tablet_manager.cc:1918] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:19.841989 31044 log.cc:1196] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-2/wal/wals/526e77e34c024571adfe16304a5167e4
I20260807 19:36:19.842684 31044 ts_tablet_manager.cc:1939] T 526e77e34c024571adfe16304a5167e4 P 7c288f5a8ff84db3bbfd1c32f2192db1: Deleting consensus metadata
I20260807 19:36:19.844897 30495 catalog_manager.cc:4841] TS 7c288f5a8ff84db3bbfd1c32f2192db1 (127.8.216.131:46753): tablet 526e77e34c024571adfe16304a5167e4 (table test-table [id=6e83b0b103e74df28007b8457e2c4347]) successfully deleted
I20260807 19:36:19.845960 31045 tablet_replica.cc:330] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: stopping tablet replica
I20260807 19:36:19.846450 31045 raft_consensus.cc:2229] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:36:19.847007 31045 raft_consensus.cc:2258] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:19.848044 31046 tablet_replica.cc:330] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: stopping tablet replica
I20260807 19:36:19.848544 31046 raft_consensus.cc:2229] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:36:19.849300 31046 raft_consensus.cc:2258] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:36:19.849866 31045 ts_tablet_manager.cc:1905] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:19.851140 31046 ts_tablet_manager.cc:1905] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:36:19.855706 31007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.831055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.8.216.129:42981)
0807 19:36:19.831299 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.831310 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.837985 (+ 6675us) server_negotiation.cc:407] Connection header received
0807 19:36:19.839209 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.839219 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.839380 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.839491 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:19.841120 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.842785 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.847495 (+ 4710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.848207 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.848348 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.853759 (+ 5411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:19.853771 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:19.853788 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:19.854673 (+ 885us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:19.854738 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:19.854770 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:19.854895 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.855168 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.855311 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:36:19.856668 31045 ts_tablet_manager.cc:1918] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:19.856887 31045 log.cc:1196] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-0/wal/wals/526e77e34c024571adfe16304a5167e4
I20260807 19:36:19.857474 31045 ts_tablet_manager.cc:1939] T 526e77e34c024571adfe16304a5167e4 P 91549ed888834d2cb5144b6cc0d3de70: Deleting consensus metadata
I20260807 19:36:19.858119 30704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:19.859534 31046 ts_tablet_manager.cc:1918] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:36:19.859726 31046 log.cc:1196] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Deleting WAL directory at /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131039239543-9058-0/minicluster-data/ts-1/wal/wals/526e77e34c024571adfe16304a5167e4
I20260807 19:36:19.860353 31046 ts_tablet_manager.cc:1939] T 526e77e34c024571adfe16304a5167e4 P 14d8fa8c50ad4db8bb6d711f205d3215: Deleting consensus metadata
I20260807 19:36:19.862869 30492 catalog_manager.cc:4841] TS 14d8fa8c50ad4db8bb6d711f205d3215 (127.8.216.130:38303): tablet 526e77e34c024571adfe16304a5167e4 (table test-table [id=6e83b0b103e74df28007b8457e2c4347]) successfully deleted
I20260807 19:36:19.970537 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30573
I20260807 19:36:19.970547 30492 catalog_manager.cc:4841] TS 91549ed888834d2cb5144b6cc0d3de70 (127.8.216.129:42981): tablet 526e77e34c024571adfe16304a5167e4 (table test-table [id=6e83b0b103e74df28007b8457e2c4347]) successfully deleted
I20260807 19:36:19.992226 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30713
I20260807 19:36:20.000375 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:19.993070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.8.216.130:38303)
0807 19:36:19.993267 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:19.993272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:19.993291 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:36:19.993502 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:19.993507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:19.993590 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:19.993681 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:19.994184 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.994967 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.997041 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:19.997507 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:19.997614 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:19.998831 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:19.998840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:19.998855 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:19.999510 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:19.999561 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:19.999582 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:19.999667 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:19.999851 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 19:36:19.999955 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:36:20.001216 30846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:20.119771 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30713
I20260807 19:36:20.145906 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30855
I20260807 19:36:20.153942 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:20.146915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.8.216.131:46753)
0807 19:36:20.147098 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:20.147103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:20.147122 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:36:20.147239 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:20.147242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:20.147342 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:20.147461 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:20.147895 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:20.148832 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:20.150739 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:20.151167 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:20.151261 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:20.152448 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:20.152458 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:20.152472 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:20.153102 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:20.153152 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:20.153172 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:20.153262 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:20.153453 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 19:36:20.153550 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:36:20.154662 30988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:20.248296 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30855
I20260807 19:36:20.269569 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 30476
I20260807 19:36:20.270860 30541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:20.386585 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 30476
2026-08-07T19:36:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3392 ms)
[----------] 6 tests from EncryptionPolicyTest (24274 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:20 dist-test-slave-8w9v krb5kdc[31055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:20 dist-test-slave-8w9v krb5kdc[31055](info): set up 2 sockets
Aug 07 19:36:20 dist-test-slave-8w9v krb5kdc[31055](info): commencing operation
krb5kdc: starting...
W20260807 19:36:21.590934 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:21 dist-test-slave-8w9v krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:21.948030 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:22.323396 31068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:22.323727 31068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:22.323827 31068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:22.331175 31068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:22.331305 31068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:22.331357 31068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:22.331421 31068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:22.331467 31068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:22.341866 31068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:22.343230 31068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:22.344677 31068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:22.357088 31073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:22.357133 31077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:22.357172 31075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:22.358695 31068 server_base.cc:1034] running on GCE node
I20260807 19:36:22.359627 31068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:22.361212 31068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:22.362455 31068 hybrid_clock.cc:648] HybridClock initialized: now 1786131382362411 us; error 53 us; skew 500 ppm
I20260807 19:36:22.362864 31068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:22 dist-test-slave-8w9v krb5kdc[31055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131382, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:22.366279 31068 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:22.367782 31068 webserver.cc:460] Webserver started at http://127.0.0.1:44089/ using document root <none> and password file <none>
I20260807 19:36:22.368607 31068 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:22.368714 31068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:22.368978 31068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:22.372036 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "9be97714060d4227ae2ffe1917077102"
format_stamp: "Formatted at 2026-08-07 19:36:22 on dist-test-slave-8w9v"
server_key: "94df3f7e42e7cf029673f8b0605c7f97"
server_key_iv: "56ccbac6aad93f775d0a9376a626c625"
server_key_version: "encryptionkey@0"
I20260807 19:36:22.372915 31068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "9be97714060d4227ae2ffe1917077102"
format_stamp: "Formatted at 2026-08-07 19:36:22 on dist-test-slave-8w9v"
server_key: "94df3f7e42e7cf029673f8b0605c7f97"
server_key_iv: "56ccbac6aad93f775d0a9376a626c625"
server_key_version: "encryptionkey@0"
I20260807 19:36:22.378366 31068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:36:22.381960 31085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:22.383359 31068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:36:22.383526 31068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "9be97714060d4227ae2ffe1917077102"
format_stamp: "Formatted at 2026-08-07 19:36:22 on dist-test-slave-8w9v"
server_key: "94df3f7e42e7cf029673f8b0605c7f97"
server_key_iv: "56ccbac6aad93f775d0a9376a626c625"
server_key_version: "encryptionkey@0"
I20260807 19:36:22.383719 31068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:22.411190 31068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:22.415771 31068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:22.416062 31068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:22.436100 31068 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42631
I20260807 19:36:22.436146 31142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42631 every 8 connection(s)
I20260807 19:36:22.437668 31068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:22.438866 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31068
I20260807 19:36:22.439139 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:22.440009 9058 external_mini_cluster.cc:1428] Setting key bef5155468cde528bc59d29a4a7655bd
I20260807 19:36:22.443001 31143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:36:22 dist-test-slave-8w9v krb5kdc[31055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:36:22.456113 31143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102: Bootstrap starting.
I20260807 19:36:22.460532 31143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:22.461974 31143 log.cc:824] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:22.465644 31143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102: No bootstrap required, opened a new log
I20260807 19:36:22.465948 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:22.442113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36513 (local address 127.0.0.1:42631)
0807 19:36:22.442929 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:22.442945 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:22.443002 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:36:22.445427 (+ 2425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:22.445455 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:22.445961 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:22.446408 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:22.447766 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:22.449930 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:22.451211 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:22.452260 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:22.452461 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:22.456782 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:22.456793 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:22.456812 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:22.456855 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:22.461101 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:22.462067 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:22.462077 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:22.462088 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:22.462315 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:22.462860 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:22.462863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:22.462865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:22.463460 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:22.463775 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:22.464236 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:36:22.464587 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":346,"threads_started":1}
I20260807 19:36:22.470090 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:22.474432 31143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } }
I20260807 19:36:22.474893 31143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:22.475067 31143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be97714060d4227ae2ffe1917077102, State: Initialized, Role: FOLLOWER
I20260807 19:36:22.475721 31143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } }
I20260807 19:36:22.476013 31143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:22.476214 31143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:22.476432 31143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:22.478730 31143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } }
I20260807 19:36:22.479285 31143 leader_election.cc:302] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9be97714060d4227ae2ffe1917077102; no voters:
I20260807 19:36:22.480019 31143 leader_election.cc:288] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:22.480173 31153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:22.480903 31153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [term 1 LEADER]: Becoming Leader. State: Replica: 9be97714060d4227ae2ffe1917077102, State: Running, Role: LEADER
I20260807 19:36:22.481592 31153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } }
I20260807 19:36:22.482540 31143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:22.486728 31155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9be97714060d4227ae2ffe1917077102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } } }
I20260807 19:36:22.487154 31155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:22.488197 31154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be97714060d4227ae2ffe1917077102. Latest consensus state: current_term: 1 leader_uuid: "9be97714060d4227ae2ffe1917077102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be97714060d4227ae2ffe1917077102" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42631 } } }
I20260807 19:36:22.488569 31154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:22.492944 31161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:22.495036 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:22.475050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59095 (local address 127.0.0.1:42631)
0807 19:36:22.475300 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:22.475310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:22.475866 (+ 556us) server_negotiation.cc:407] Connection header received
0807 19:36:22.476063 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:22.476067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:22.476183 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:22.476302 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:22.477794 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:22.478557 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:22.479915 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:22.480809 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:22.480999 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:22.485111 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:22.485119 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:22.485123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:22.485194 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:22.489699 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:22.490830 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:22.490836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:22.490839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:22.491068 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:22.491820 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:22.491828 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:22.491835 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:22.492304 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:22.493893 (+ 1589us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:22.494122 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 19:36:22.494340 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:36:22.499397 31161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:22.509243 31161 catalog_manager.cc:1347] Generated new cluster ID: 6e3a3a46e819477da70ebc70b5c11128
I20260807 19:36:22.509379 31161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:22.531982 31161 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:22.534082 31161 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:22.547187 31161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9be97714060d4227ae2ffe1917077102: Generated new TSK 0
I20260807 19:36:22.548185 31161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:22.561443 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31068
I20260807 19:36:22.563223 31138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:22.658407 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31068
W20260807 19:36:22.679661 31053 connection.cc:535] client connection to 127.0.0.1:42631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42631): Connection reset by peer (error 104)
2026-08-07T19:36:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:22 dist-test-slave-8w9v krb5kdc[31178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:22 dist-test-slave-8w9v krb5kdc[31178](info): set up 2 sockets
Aug 07 19:36:22 dist-test-slave-8w9v krb5kdc[31178](info): commencing operation
krb5kdc: starting...
W20260807 19:36:23.847890 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:24 dist-test-slave-8w9v krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:24.200350 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:24.565109 31192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:24.565516 31192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:24.565688 31192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:24.573173 31192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:24.573350 31192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:24.573462 31192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:24.573573 31192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:24.573659 31192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:24.583956 31192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41457
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:24.585418 31192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:24.586824 31192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:24.599543 31200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:24.599491 31197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:24.599478 31198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:24.600672 31192 server_base.cc:1034] running on GCE node
I20260807 19:36:24.601594 31192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:24.603426 31192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:24.604707 31192 hybrid_clock.cc:648] HybridClock initialized: now 1786131384604660 us; error 68 us; skew 500 ppm
I20260807 19:36:24.605121 31192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:24 dist-test-slave-8w9v krb5kdc[31178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131384, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:24.608888 31192 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:24.610710 31192 webserver.cc:460] Webserver started at http://127.0.0.1:35533/ using document root <none> and password file <none>
I20260807 19:36:24.611608 31192 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:24.611786 31192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:24.612208 31192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:24.615774 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "211a71c5a0ae4ccf8e0617c8f1a64633"
format_stamp: "Formatted at 2026-08-07 19:36:24 on dist-test-slave-8w9v"
server_key: "fc52f29ad7bfbbf15c5eadf75aace0af"
server_key_iv: "d2eb743e611b204615bb1616665e7a1b"
server_key_version: "encryptionkey@0"
I20260807 19:36:24.616787 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "211a71c5a0ae4ccf8e0617c8f1a64633"
format_stamp: "Formatted at 2026-08-07 19:36:24 on dist-test-slave-8w9v"
server_key: "fc52f29ad7bfbbf15c5eadf75aace0af"
server_key_iv: "d2eb743e611b204615bb1616665e7a1b"
server_key_version: "encryptionkey@0"
I20260807 19:36:24.623010 31192 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 19:36:24.627327 31210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:24.628976 31192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:36:24.629218 31192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "211a71c5a0ae4ccf8e0617c8f1a64633"
format_stamp: "Formatted at 2026-08-07 19:36:24 on dist-test-slave-8w9v"
server_key: "fc52f29ad7bfbbf15c5eadf75aace0af"
server_key_iv: "d2eb743e611b204615bb1616665e7a1b"
server_key_version: "encryptionkey@0"
I20260807 19:36:24.629482 31192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:24.661612 31192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:24.666353 31192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:24.666699 31192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:24.688689 31192 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41457
I20260807 19:36:24.688712 31267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41457 every 8 connection(s)
I20260807 19:36:24.690315 31192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:24.694221 31268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:24.695765 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31192
I20260807 19:36:24.695995 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:24.696617 9058 external_mini_cluster.cc:1428] Setting key d678d8b0fd9591db767487dd7086ca85
I20260807 19:36:24.709141 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633: Bootstrap starting.
I20260807 19:36:24.710534 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:24.698501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40969 (local address 127.0.0.1:41457)
0807 19:36:24.699353 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:24.699365 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:24.699411 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:36:24.701177 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:24.701206 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:24.701639 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:24.701994 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:24.702625 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:24.703918 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:24.705920 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:24.706658 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:24.706794 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:24.708232 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:24.708241 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:24.708256 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:24.708806 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:24.708884 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:24.708921 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:24.709042 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:24.709390 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:36:24.709536 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":398,"threads_started":1}
I20260807 19:36:24.712683 31268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:24.713869 31268 log.cc:824] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:24.714476 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:24.716703 31268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633: No bootstrap required, opened a new log
I20260807 19:36:24.724869 31268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } }
I20260807 19:36:24.725364 31268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:24.725476 31268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 211a71c5a0ae4ccf8e0617c8f1a64633, State: Initialized, Role: FOLLOWER
I20260807 19:36:24.726138 31268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } }
I20260807 19:36:24.726383 31268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:24.726536 31268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:24.726712 31268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:24.729554 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:24.722587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40569 (local address 127.0.0.1:41457)
0807 19:36:24.722796 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:24.722803 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:24.723213 (+ 410us) server_negotiation.cc:407] Connection header received
0807 19:36:24.723421 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:24.723427 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:24.723510 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:24.723617 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:24.724108 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:24.725232 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:24.726766 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:24.727487 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:24.727631 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:24.728769 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:24.728775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:24.728779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:24.728800 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:24.728873 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:24.728900 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:24.729018 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:24.729172 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 19:36:24.729229 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:36:24.729959 31268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } }
I20260807 19:36:24.730562 31268 leader_election.cc:302] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 211a71c5a0ae4ccf8e0617c8f1a64633; no voters:
I20260807 19:36:24.731269 31268 leader_election.cc:288] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:24.731392 31279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:24.732221 31279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [term 1 LEADER]: Becoming Leader. State: Replica: 211a71c5a0ae4ccf8e0617c8f1a64633, State: Running, Role: LEADER
I20260807 19:36:24.732784 31279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } }
I20260807 19:36:24.733603 31268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:24.736192 31281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 211a71c5a0ae4ccf8e0617c8f1a64633. Latest consensus state: current_term: 1 leader_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } } }
I20260807 19:36:24.736713 31281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:24.736908 31280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "211a71c5a0ae4ccf8e0617c8f1a64633" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41457 } } }
I20260807 19:36:24.737327 31280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:24.737742 31288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:24.744330 31288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:24.754812 31288 catalog_manager.cc:1347] Generated new cluster ID: 7902fb87ad3b45d9a24b449797a1f024
I20260807 19:36:24.755030 31288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:24.767887 31288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:24.769867 31288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:24.780092 31288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 211a71c5a0ae4ccf8e0617c8f1a64633: Generated new TSK 0
I20260807 19:36:24.781011 31288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:24.793238 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31192
W20260807 19:36:24.794742 31263 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40969
W20260807 19:36:24.795715 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31192: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:24.795930 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31192
W20260807 19:36:24.817066 31175 connection.cc:535] client connection to 127.0.0.1:41457 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41457): Connection reset by peer (error 104)
2026-08-07T19:36:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:24 dist-test-slave-8w9v krb5kdc[31303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:24 dist-test-slave-8w9v krb5kdc[31303](info): set up 2 sockets
Aug 07 19:36:24 dist-test-slave-8w9v krb5kdc[31303](info): commencing operation
krb5kdc: starting...
W20260807 19:36:25.984313 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:26 dist-test-slave-8w9v krb5kdc[31303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:26.325742 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:26.682144 31317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:26.682457 31317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:26.682559 31317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:26.689958 31317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:26.690088 31317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:26.690136 31317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:26.690193 31317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:26.690240 31317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:26.700415 31317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39629
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:26.701752 31317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:26.703074 31317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:26.715260 31323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:26.715310 31322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:26.715334 31325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:26.721199 31317 server_base.cc:1034] running on GCE node
I20260807 19:36:26.722064 31317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:26.723582 31317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:26.724800 31317 hybrid_clock.cc:648] HybridClock initialized: now 1786131386724742 us; error 57 us; skew 500 ppm
I20260807 19:36:26.725155 31317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:26 dist-test-slave-8w9v krb5kdc[31303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:26.728385 31317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:26.729933 31317 webserver.cc:460] Webserver started at http://127.0.0.1:33807/ using document root <none> and password file <none>
I20260807 19:36:26.730672 31317 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:26.730813 31317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:26.731125 31317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:26.734259 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "eba3857a1adc49b499982a1db7499d34"
format_stamp: "Formatted at 2026-08-07 19:36:26 on dist-test-slave-8w9v"
server_key: "2e2c36ff1accd8384f57d0f92084e0dd"
server_key_iv: "80a3200af2444316c4b225962d2c7f0b"
server_key_version: "encryptionkey@0"
I20260807 19:36:26.735049 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "eba3857a1adc49b499982a1db7499d34"
format_stamp: "Formatted at 2026-08-07 19:36:26 on dist-test-slave-8w9v"
server_key: "2e2c36ff1accd8384f57d0f92084e0dd"
server_key_iv: "80a3200af2444316c4b225962d2c7f0b"
server_key_version: "encryptionkey@0"
I20260807 19:36:26.740345 31317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:36:26.743799 31336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:26.745168 31317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:26.745357 31317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "eba3857a1adc49b499982a1db7499d34"
format_stamp: "Formatted at 2026-08-07 19:36:26 on dist-test-slave-8w9v"
server_key: "2e2c36ff1accd8384f57d0f92084e0dd"
server_key_iv: "80a3200af2444316c4b225962d2c7f0b"
server_key_version: "encryptionkey@0"
I20260807 19:36:26.745582 31317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:26.751493 31317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:26.755570 31317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:26.755872 31317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:26.776084 31317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39629
I20260807 19:36:26.776173 31394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39629 every 8 connection(s)
I20260807 19:36:26.777598 31317 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:26.779767 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31317
I20260807 19:36:26.779955 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:26.780462 9058 external_mini_cluster.cc:1428] Setting key 04061cd530e6f212657dfad30aaecaf7
I20260807 19:36:26.781876 31395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:26.789417 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:26.782285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37921 (local address 127.0.0.1:39629)
0807 19:36:26.783107 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:26.783124 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:26.783178 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:36:26.785331 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:26.785361 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:26.785831 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:26.786303 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:26.787317 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:26.787327 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:26.787344 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:26.787361 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:26.787467 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:26.787493 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:26.787707 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:26.787847 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 19:36:26.788072 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":330,"threads_started":1}
I20260807 19:36:26.793684 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:26.798228 31395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34: Bootstrap starting.
I20260807 19:36:26.801982 31395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:26.802346 31400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:26.799421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46331 (local address 127.0.0.1:39629)
0807 19:36:26.799622 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:26.799628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:26.800117 (+ 489us) server_negotiation.cc:407] Connection header received
0807 19:36:26.800356 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:26.800370 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:26.800434 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:26.800538 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:26.801540 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:26.801546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:26.801551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:26.801571 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:26.801622 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:26.801638 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:26.801760 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:26.801888 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:36:26.801956 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:36:26.803524 31395 log.cc:824] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:26.806699 31395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34: No bootstrap required, opened a new log
I20260807 19:36:26.814545 31395 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } }
I20260807 19:36:26.814975 31395 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:26.815081 31395 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba3857a1adc49b499982a1db7499d34, State: Initialized, Role: FOLLOWER
I20260807 19:36:26.815758 31395 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } }
I20260807 19:36:26.816030 31395 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:26.816207 31395 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:26.816394 31395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:26.818683 31395 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } }
I20260807 19:36:26.819216 31395 leader_election.cc:302] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba3857a1adc49b499982a1db7499d34; no voters:
I20260807 19:36:26.819969 31395 leader_election.cc:288] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:26.820086 31408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:26.821115 31408 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [term 1 LEADER]: Becoming Leader. State: Replica: eba3857a1adc49b499982a1db7499d34, State: Running, Role: LEADER
I20260807 19:36:26.821892 31408 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } }
I20260807 19:36:26.823084 31395 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:26.825007 31409 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba3857a1adc49b499982a1db7499d34. Latest consensus state: current_term: 1 leader_uuid: "eba3857a1adc49b499982a1db7499d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } } }
I20260807 19:36:26.825467 31409 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:26.826725 31410 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba3857a1adc49b499982a1db7499d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba3857a1adc49b499982a1db7499d34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39629 } } }
I20260807 19:36:26.827144 31410 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:26.828347 31415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:26.833133 31415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:26.841976 31415 catalog_manager.cc:1347] Generated new cluster ID: eb17d5bc989448118e4075bec691f68f
I20260807 19:36:26.842113 31415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:26.857000 31415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:26.857582 31415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:26.877913 31415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba3857a1adc49b499982a1db7499d34: Generated new TSK 0
I20260807 19:36:26.879069 31415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:26.885576 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31317
W20260807 19:36:26.886835 31390 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37921
W20260807 19:36:26.887607 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31317: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:26.887775 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31317
2026-08-07T19:36:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2089 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:27 dist-test-slave-8w9v krb5kdc[31433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:27 dist-test-slave-8w9v krb5kdc[31433](info): set up 2 sockets
Aug 07 19:36:27 dist-test-slave-8w9v krb5kdc[31433](info): commencing operation
krb5kdc: starting...
W20260807 19:36:28.083343 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:28 dist-test-slave-8w9v krb5kdc[31433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:28.424302 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:28.783932 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:28.784391 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:28.784587 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:28.792444 31446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:28.792622 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:28.792729 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:28.792850 31446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:28.792936 31446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:28.803637 31446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:28.805229 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:28.806631 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:28.819243 31451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:28.819375 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:28.819275 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:28.820461 31446 server_base.cc:1034] running on GCE node
I20260807 19:36:28.821288 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:28.822777 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:28.824015 31446 hybrid_clock.cc:648] HybridClock initialized: now 1786131388823966 us; error 42 us; skew 500 ppm
I20260807 19:36:28.824436 31446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:28 dist-test-slave-8w9v krb5kdc[31433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131388, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:28.827924 31446 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:28.829689 31446 webserver.cc:460] Webserver started at http://127.0.0.1:38577/ using document root <none> and password file <none>
I20260807 19:36:28.830442 31446 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:28.830560 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:28.830845 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:28.834280 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "a84bade6f3c84a3a8c7cccd7324341b8"
format_stamp: "Formatted at 2026-08-07 19:36:28 on dist-test-slave-8w9v"
server_key: "f7c22c629e311d512ad0a48f9a09d36c"
server_key_iv: "d16d46c1bba9f910976bd0121ee8e095"
server_key_version: "encryptionkey@0"
I20260807 19:36:28.835098 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "a84bade6f3c84a3a8c7cccd7324341b8"
format_stamp: "Formatted at 2026-08-07 19:36:28 on dist-test-slave-8w9v"
server_key: "f7c22c629e311d512ad0a48f9a09d36c"
server_key_iv: "d16d46c1bba9f910976bd0121ee8e095"
server_key_version: "encryptionkey@0"
I20260807 19:36:28.841157 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:36:28.845198 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:28.846763 31446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:36:28.846987 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "a84bade6f3c84a3a8c7cccd7324341b8"
format_stamp: "Formatted at 2026-08-07 19:36:28 on dist-test-slave-8w9v"
server_key: "f7c22c629e311d512ad0a48f9a09d36c"
server_key_iv: "d16d46c1bba9f910976bd0121ee8e095"
server_key_version: "encryptionkey@0"
I20260807 19:36:28.847263 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:28.860047 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:28.865149 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:28.865496 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:28.887024 31446 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44959
I20260807 19:36:28.887019 31516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44959 every 8 connection(s)
I20260807 19:36:28.888733 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:28.892931 31518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:28.897497 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31446
I20260807 19:36:28.897728 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:28.898166 9058 external_mini_cluster.cc:1428] Setting key dde80648b41b377b00fa8ea5b023f946
I20260807 19:36:28.904850 31518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8: Bootstrap starting.
I20260807 19:36:28.908845 31518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:28.910084 31518 log.cc:824] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:28.914026 31518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8: No bootstrap required, opened a new log
Aug 07 19:36:28 dist-test-slave-8w9v krb5kdc[31433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:36:28.925457 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:28.900090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52997 (local address 127.0.0.1:44959)
0807 19:36:28.901008 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:28.901020 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:28.901065 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:36:28.903197 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:28.903224 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:28.903652 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:28.903990 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:28.906138 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:28.907519 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:28.909434 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:28.910164 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:28.910388 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:28.915586 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:28.915598 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:28.915620 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:28.915682 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:28.920196 (+ 4514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:28.921039 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:28.921050 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:28.921065 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:28.921310 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:28.921715 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:28.921720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:28.921724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:28.922565 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:28.922911 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:28.923284 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 19:36:28.923858 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":408,"threads_started":1}
I20260807 19:36:28.925735 31518 raft_consensus.cc:348] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } }
I20260807 19:36:28.926198 31518 raft_consensus.cc:374] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:28.926337 31518 raft_consensus.cc:729] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84bade6f3c84a3a8c7cccd7324341b8, State: Initialized, Role: FOLLOWER
I20260807 19:36:28.926939 31518 consensus_queue.cc:260] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } }
I20260807 19:36:28.927224 31518 raft_consensus.cc:388] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:28.927377 31518 raft_consensus.cc:482] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:28.927584 31518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:28.929862 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:28.930506 31518 raft_consensus.cc:504] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } }
I20260807 19:36:28.931121 31518 leader_election.cc:302] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a84bade6f3c84a3a8c7cccd7324341b8; no voters:
I20260807 19:36:28.931788 31518 leader_election.cc:288] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:28.931952 31523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:28.933490 31523 raft_consensus.cc:686] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [term 1 LEADER]: Becoming Leader. State: Replica: a84bade6f3c84a3a8c7cccd7324341b8, State: Running, Role: LEADER
I20260807 19:36:28.934098 31523 consensus_queue.cc:237] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } }
I20260807 19:36:28.934582 31518 sys_catalog.cc:564] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:28.936951 31524 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } } }
I20260807 19:36:28.937384 31524 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:28.937942 31525 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a84bade6f3c84a3a8c7cccd7324341b8. Latest consensus state: current_term: 1 leader_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84bade6f3c84a3a8c7cccd7324341b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44959 } } }
I20260807 19:36:28.938235 31525 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:28.940256 31531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:28.947865 31531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:28.962441 31531 catalog_manager.cc:1347] Generated new cluster ID: 031dd7c2bd29414998c58d34524ba0b7
I20260807 19:36:28.962574 31531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:28.968868 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:28.942748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53575 (local address 127.0.0.1:44959)
0807 19:36:28.942986 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:28.942996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:28.949892 (+ 6896us) server_negotiation.cc:407] Connection header received
0807 19:36:28.950185 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:28.950191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:28.950293 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:28.950400 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:28.953027 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:28.954340 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:28.956316 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:28.956946 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:28.957047 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:28.960667 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:28.960673 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:28.960677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:28.960719 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:28.964821 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:28.965750 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:28.965755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:28.965759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:28.965905 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:28.966323 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:28.966328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:28.966331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:28.966734 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:28.966974 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:28.968260 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0807 19:36:28.968502 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:36:28.994716 31531 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:28.996883 31531 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:29.011255 31531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a84bade6f3c84a3a8c7cccd7324341b8: Generated new TSK 0
I20260807 19:36:29.012305 31531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:29.030203 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31446
I20260807 19:36:29.031665 31512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:29.129529 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31446
2026-08-07T19:36:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2245 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:29 dist-test-slave-8w9v krb5kdc[31551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:29 dist-test-slave-8w9v krb5kdc[31551](info): set up 2 sockets
Aug 07 19:36:29 dist-test-slave-8w9v krb5kdc[31551](info): commencing operation
krb5kdc: starting...
W20260807 19:36:30.328190 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:30 dist-test-slave-8w9v krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:30.671109 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:31.035342 31567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:31.035668 31567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:31.035779 31567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:31.043413 31567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:31.043541 31567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:31.043592 31567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:31.043658 31567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:31.043704 31567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:31.054232 31567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:31.055605 31567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:31.057047 31567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:31.069892 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:31.069959 31575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:31.069860 31573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:31.071712 31567 server_base.cc:1034] running on GCE node
I20260807 19:36:31.072690 31567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:31.074232 31567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:31.075459 31567 hybrid_clock.cc:648] HybridClock initialized: now 1786131391075415 us; error 41 us; skew 500 ppm
I20260807 19:36:31.075855 31567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:31 dist-test-slave-8w9v krb5kdc[31551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:31.085393 31567 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:31.087165 31567 webserver.cc:460] Webserver started at http://127.0.0.1:40561/ using document root <none> and password file <none>
I20260807 19:36:31.087961 31567 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:31.088166 31567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:31.088536 31567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:31.091691 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "15a8213fbd414145845485b2bc8f4d33"
format_stamp: "Formatted at 2026-08-07 19:36:31 on dist-test-slave-8w9v"
server_key: "aa64f042f826a2e89484c00734ee2629"
server_key_iv: "e048facb8ddd7bfe864786da1664e29d"
server_key_version: "encryptionkey@0"
I20260807 19:36:31.092558 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "15a8213fbd414145845485b2bc8f4d33"
format_stamp: "Formatted at 2026-08-07 19:36:31 on dist-test-slave-8w9v"
server_key: "aa64f042f826a2e89484c00734ee2629"
server_key_iv: "e048facb8ddd7bfe864786da1664e29d"
server_key_version: "encryptionkey@0"
I20260807 19:36:31.098342 31567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:36:31.102247 31584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:31.103638 31567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:36:31.103844 31567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "15a8213fbd414145845485b2bc8f4d33"
format_stamp: "Formatted at 2026-08-07 19:36:31 on dist-test-slave-8w9v"
server_key: "aa64f042f826a2e89484c00734ee2629"
server_key_iv: "e048facb8ddd7bfe864786da1664e29d"
server_key_version: "encryptionkey@0"
I20260807 19:36:31.104063 31567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:31.118623 31567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:31.123500 31567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:31.123842 31567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:31.143775 31567 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39677
I20260807 19:36:31.143798 31637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39677 every 8 connection(s)
I20260807 19:36:31.145324 31567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:31.146811 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31567
I20260807 19:36:31.147022 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:31.147590 9058 external_mini_cluster.cc:1428] Setting key 804eda68d20c88c2beaeea2d1ec40c03
I20260807 19:36:31.150260 31638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:31.161556 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:31.149708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34629 (local address 127.0.0.1:39677)
0807 19:36:31.150657 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:31.150670 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:31.150713 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:36:31.152465 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:31.152487 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:31.152961 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:31.153389 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:31.153967 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:31.155347 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:31.156854 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:31.157578 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:31.157792 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:31.159067 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:31.159075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:31.159093 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:31.159696 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:31.159784 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:31.159818 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:31.159957 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:31.160234 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:36:31.160552 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":420,"threads_started":1}
I20260807 19:36:31.162138 31638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33: Bootstrap starting.
I20260807 19:36:31.165515 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:31.165656 31638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:31.166698 31638 log.cc:824] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:31.169801 31638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33: No bootstrap required, opened a new log
I20260807 19:36:31.177412 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:31.170239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55273 (local address 127.0.0.1:39677)
0807 19:36:31.170517 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:31.170524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:31.170982 (+ 458us) server_negotiation.cc:407] Connection header received
0807 19:36:31.171207 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:31.171213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:31.171303 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:31.171452 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:31.171966 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:31.173174 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:31.174557 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:31.175308 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:31.175446 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:31.176584 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:31.176595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:31.176600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:31.176624 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:31.176675 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:31.176701 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:31.176800 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:31.176938 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 19:36:31.177042 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:36:31.180060 31638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } }
I20260807 19:36:31.180554 31638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:31.180639 31638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a8213fbd414145845485b2bc8f4d33, State: Initialized, Role: FOLLOWER
I20260807 19:36:31.181190 31638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } }
I20260807 19:36:31.181411 31638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:31.181538 31638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:31.181675 31638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:31.184338 31638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } }
I20260807 19:36:31.184964 31638 leader_election.cc:302] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15a8213fbd414145845485b2bc8f4d33; no voters:
I20260807 19:36:31.185602 31638 leader_election.cc:288] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:31.185772 31650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:31.186583 31650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [term 1 LEADER]: Becoming Leader. State: Replica: 15a8213fbd414145845485b2bc8f4d33, State: Running, Role: LEADER
I20260807 19:36:31.187194 31650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } }
I20260807 19:36:31.187956 31638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:31.190112 31651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15a8213fbd414145845485b2bc8f4d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } } }
I20260807 19:36:31.190515 31651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:31.191282 31652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15a8213fbd414145845485b2bc8f4d33. Latest consensus state: current_term: 1 leader_uuid: "15a8213fbd414145845485b2bc8f4d33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15a8213fbd414145845485b2bc8f4d33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39677 } } }
I20260807 19:36:31.192467 31652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:31.191439 31655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:31.198077 31655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:31.207818 31655 catalog_manager.cc:1347] Generated new cluster ID: 3947ba46f1c540e2a1ef18fcc68a1999
I20260807 19:36:31.207988 31655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:31.222921 31655 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:31.224396 31655 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:31.232625 31655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15a8213fbd414145845485b2bc8f4d33: Generated new TSK 0
I20260807 19:36:31.233666 31655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:31.247627 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31567
W20260807 19:36:31.249105 31633 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34629
W20260807 19:36:31.249914 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31567: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:31.250062 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31567
2026-08-07T19:36:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2118 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:31 dist-test-slave-8w9v krb5kdc[31674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:31 dist-test-slave-8w9v krb5kdc[31674](info): set up 2 sockets
Aug 07 19:36:31 dist-test-slave-8w9v krb5kdc[31674](info): commencing operation
krb5kdc: starting...
W20260807 19:36:32.443373 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:32 dist-test-slave-8w9v krb5kdc[31674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:32.798966 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:33.172220 31687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:33.172603 31687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:33.172786 31687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:33.180763 31687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:33.181058 31687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:33.181190 31687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:33.181354 31687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:33.181452 31687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:33.192837 31687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:33.194339 31687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:33.195726 31687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:33.208411 31694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:33.208451 31692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:33.209329 31696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:33.209650 31687 server_base.cc:1034] running on GCE node
I20260807 19:36:33.210564 31687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:33.212232 31687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:33.213490 31687 hybrid_clock.cc:648] HybridClock initialized: now 1786131393213430 us; error 55 us; skew 500 ppm
I20260807 19:36:33.213900 31687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:33 dist-test-slave-8w9v krb5kdc[31674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:33.217298 31687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:33.218923 31687 webserver.cc:460] Webserver started at http://127.0.0.1:35595/ using document root <none> and password file <none>
I20260807 19:36:33.219683 31687 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:33.219861 31687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:33.220254 31687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:33.223409 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c"
format_stamp: "Formatted at 2026-08-07 19:36:33 on dist-test-slave-8w9v"
server_key: "6bc6ed8fff452c2fec5235b2e5c49802"
server_key_iv: "9368872b1df86dccc1c28f1419f682bb"
server_key_version: "encryptionkey@0"
I20260807 19:36:33.224320 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c"
format_stamp: "Formatted at 2026-08-07 19:36:33 on dist-test-slave-8w9v"
server_key: "6bc6ed8fff452c2fec5235b2e5c49802"
server_key_iv: "9368872b1df86dccc1c28f1419f682bb"
server_key_version: "encryptionkey@0"
I20260807 19:36:33.229766 31687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 19:36:33.233654 31704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:33.235142 31687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:33.235356 31687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c"
format_stamp: "Formatted at 2026-08-07 19:36:33 on dist-test-slave-8w9v"
server_key: "6bc6ed8fff452c2fec5235b2e5c49802"
server_key_iv: "9368872b1df86dccc1c28f1419f682bb"
server_key_version: "encryptionkey@0"
I20260807 19:36:33.235587 31687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:33.241670 31687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:33.246052 31687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:33.246389 31687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:33.266448 31687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34311
I20260807 19:36:33.266500 31760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34311 every 8 connection(s)
I20260807 19:36:33.268002 31687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:33.271781 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:33.273505 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31687
I20260807 19:36:33.273703 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:33.274320 9058 external_mini_cluster.cc:1428] Setting key 41ecc7a5d56f0605c6781f98cfeeb228
I20260807 19:36:33.282256 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:33.276244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35017 (local address 127.0.0.1:34311)
0807 19:36:33.277112 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:33.277124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:33.277166 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:36:33.278819 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:33.278841 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:33.279189 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:33.279526 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:33.280656 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:33.280662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:33.280674 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:33.280688 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:33.280750 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:33.280769 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:33.280947 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:33.281082 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 19:36:33.281328 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":421,"threads_started":1}
I20260807 19:36:33.285802 31761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c: Bootstrap starting.
I20260807 19:36:33.286124 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:33.289976 31761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:33.291519 31761 log.cc:824] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:33.294451 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:33.291741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44245 (local address 127.0.0.1:34311)
0807 19:36:33.291934 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:33.291941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:33.292458 (+ 517us) server_negotiation.cc:407] Connection header received
0807 19:36:33.292621 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:33.292629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:33.292699 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:33.292837 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:33.293761 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:33.293769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:33.293778 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:33.293794 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:33.293835 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:33.293858 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:33.293982 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:33.294110 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:36:33.294176 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:36:33.295082 31761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c: No bootstrap required, opened a new log
I20260807 19:36:33.302948 31761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260807 19:36:33.303372 31761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:33.303494 31761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e9ac6db35d14ef6964c549f3ae9ef7c, State: Initialized, Role: FOLLOWER
I20260807 19:36:33.304239 31761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260807 19:36:33.304544 31761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:33.304708 31761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:33.304898 31761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:33.307176 31761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260807 19:36:33.307714 31761 leader_election.cc:302] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e9ac6db35d14ef6964c549f3ae9ef7c; no voters:
I20260807 19:36:33.308364 31761 leader_election.cc:288] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:33.308504 31771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:33.309332 31771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [term 1 LEADER]: Becoming Leader. State: Replica: 6e9ac6db35d14ef6964c549f3ae9ef7c, State: Running, Role: LEADER
I20260807 19:36:33.310065 31771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260807 19:36:33.310905 31761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:33.313253 31772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } } }
I20260807 19:36:33.313277 31773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e9ac6db35d14ef6964c549f3ae9ef7c. Latest consensus state: current_term: 1 leader_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e9ac6db35d14ef6964c549f3ae9ef7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } } }
I20260807 19:36:33.313751 31772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:33.313751 31773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:33.316181 31777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:33.321249 31777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:33.330170 31777 catalog_manager.cc:1347] Generated new cluster ID: c43988e45862485fb1da0826fe3e1c67
I20260807 19:36:33.330312 31777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:33.342793 31777 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:33.343276 31777 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:33.350888 31777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e9ac6db35d14ef6964c549f3ae9ef7c: Generated new TSK 0
I20260807 19:36:33.351764 31777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:33.363360 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31687
W20260807 19:36:33.364689 31756 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35017
W20260807 19:36:33.365535 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:33.365715 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31687
2026-08-07T19:36:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:33 dist-test-slave-8w9v krb5kdc[31793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:33 dist-test-slave-8w9v krb5kdc[31793](info): set up 2 sockets
Aug 07 19:36:33 dist-test-slave-8w9v krb5kdc[31793](info): commencing operation
krb5kdc: starting...
W20260807 19:36:34.551676 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:34 dist-test-slave-8w9v krb5kdc[31793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:34.913062 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:35.282366 31806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:35.282750 31806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:35.282933 31806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:35.290647 31806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:35.290839 31806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:35.290967 31806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:35.291096 31806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:35.291178 31806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:35.301918 31806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:35.303470 31806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:35.304987 31806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:35.317837 31816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:35.317941 31813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:35.317891 31812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:35.319239 31806 server_base.cc:1034] running on GCE node
I20260807 19:36:35.320165 31806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:35.321775 31806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:35.323026 31806 hybrid_clock.cc:648] HybridClock initialized: now 1786131395322964 us; error 51 us; skew 500 ppm
I20260807 19:36:35.323411 31806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:35 dist-test-slave-8w9v krb5kdc[31793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:35.326778 31806 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:35.328517 31806 webserver.cc:460] Webserver started at http://127.0.0.1:35995/ using document root <none> and password file <none>
I20260807 19:36:35.329300 31806 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:35.329453 31806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:35.329769 31806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:35.332981 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "a7ec13c4cd974b979cbbad88d6c84f77"
format_stamp: "Formatted at 2026-08-07 19:36:35 on dist-test-slave-8w9v"
server_key: "e9cce2939e859ebe1318fcd78598abc7"
server_key_iv: "869dd9cfbde426591fc0f939feca0a7a"
server_key_version: "encryptionkey@0"
I20260807 19:36:35.333786 31806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "a7ec13c4cd974b979cbbad88d6c84f77"
format_stamp: "Formatted at 2026-08-07 19:36:35 on dist-test-slave-8w9v"
server_key: "e9cce2939e859ebe1318fcd78598abc7"
server_key_iv: "869dd9cfbde426591fc0f939feca0a7a"
server_key_version: "encryptionkey@0"
I20260807 19:36:35.339332 31806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:36:35.343099 31823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:35.344681 31806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:35.344897 31806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "a7ec13c4cd974b979cbbad88d6c84f77"
format_stamp: "Formatted at 2026-08-07 19:36:35 on dist-test-slave-8w9v"
server_key: "e9cce2939e859ebe1318fcd78598abc7"
server_key_iv: "869dd9cfbde426591fc0f939feca0a7a"
server_key_version: "encryptionkey@0"
I20260807 19:36:35.345122 31806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:35.360559 31806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:35.365356 31806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:35.365674 31806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:35.387094 31806 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44387
I20260807 19:36:35.387135 31879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44387 every 8 connection(s)
I20260807 19:36:35.388731 31806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:35.392936 31880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:35.398953 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31806
I20260807 19:36:35.399156 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:35.399621 9058 external_mini_cluster.cc:1428] Setting key c3e6c8b9b4afb4943932d6fdafb281ed
I20260807 19:36:35.405189 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77: Bootstrap starting.
I20260807 19:36:35.409572 31880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:35.410964 31880 log.cc:824] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:35.414568 31880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77: No bootstrap required, opened a new log
Aug 07 19:36:35 dist-test-slave-8w9v krb5kdc[31793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:36:35.425235 31880 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } }
I20260807 19:36:35.425817 31880 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:35.426008 31880 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ec13c4cd974b979cbbad88d6c84f77, State: Initialized, Role: FOLLOWER
I20260807 19:36:35.426779 31880 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } }
I20260807 19:36:35.427107 31880 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:35.427268 31880 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:35.427496 31880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:35.430246 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:35.401995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:58987 (local address 127.0.0.1:44387)
0807 19:36:35.403144 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:35.403164 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:35.404810 (+ 1646us) server_negotiation.cc:407] Connection header received
0807 19:36:35.407555 (+ 2745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:35.407587 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:35.408181 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:35.408663 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:35.410201 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:35.412214 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:35.413763 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:35.414830 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:35.415031 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:35.419780 (+ 4749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:35.419792 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:35.419815 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:35.419873 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:35.424469 (+ 4596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:35.425522 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:35.425531 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:35.425541 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:35.425774 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:35.426521 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:35.426526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:35.426529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:35.427355 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:35.427742 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:35.428283 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 19:36:35.428722 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":498,"threads_started":1}
I20260807 19:36:35.431094 31880 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } }
I20260807 19:36:35.431667 31880 leader_election.cc:302] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ec13c4cd974b979cbbad88d6c84f77; no voters:
I20260807 19:36:35.432343 31880 leader_election.cc:288] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:35.432510 31885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:35.434098 31885 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [term 1 LEADER]: Becoming Leader. State: Replica: a7ec13c4cd974b979cbbad88d6c84f77, State: Running, Role: LEADER
I20260807 19:36:35.434791 31880 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:35.434792 31885 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } }
I20260807 19:36:35.435410 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:35.438200 31887 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ec13c4cd974b979cbbad88d6c84f77. Latest consensus state: current_term: 1 leader_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } } }
I20260807 19:36:35.438632 31887 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:35.445530 31886 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ec13c4cd974b979cbbad88d6c84f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44387 } } }
I20260807 19:36:35.446269 31886 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:35.450114 31891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:35.457630 31891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:35.482544 31891 catalog_manager.cc:1347] Generated new cluster ID: c37621dd3cf446519a98affe43726efa
I20260807 19:36:35.482699 31891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:35.499071 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:35.463882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:49955 (local address 127.0.0.1:44387)
0807 19:36:35.464078 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:35.464085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:35.480018 (+ 15933us) server_negotiation.cc:407] Connection header received
0807 19:36:35.480406 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:35.480413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:35.480501 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:35.480592 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:35.482920 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:35.484089 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:35.485997 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:35.486687 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:35.486783 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:35.490741 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:35.490749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:35.490754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:35.490799 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:35.494674 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:35.495927 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:35.495935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:35.495942 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:35.496150 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:35.496802 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:35.496808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:35.496811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:35.497181 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:35.497445 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:35.498453 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0807 19:36:35.498660 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:36:35.499641 31891 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:35.502064 31891 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:35.519835 31891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ec13c4cd974b979cbbad88d6c84f77: Generated new TSK 0
I20260807 19:36:35.520852 31891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:35.531909 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31806
I20260807 19:36:35.533459 31875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:35.629825 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31806
W20260807 19:36:35.652297 31790 connection.cc:535] client connection to 127.0.0.1:44387 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44387): Connection reset by peer (error 104)
2026-08-07T19:36:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:35 dist-test-slave-8w9v krb5kdc[31914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:35 dist-test-slave-8w9v krb5kdc[31914](info): set up 2 sockets
Aug 07 19:36:35 dist-test-slave-8w9v krb5kdc[31914](info): commencing operation
krb5kdc: starting...
W20260807 19:36:36.817827 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:37 dist-test-slave-8w9v krb5kdc[31914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:37.164021 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:37.546005 31927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:37.546368 31927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:37.546530 31927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:37.554459 31927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:37.554621 31927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:37.554733 31927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:37.554886 31927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:37.554955 31927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:37.566032 31927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:37.567539 31927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:37.569200 31927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:37.583287 31932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:37.583283 31935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:37.583398 31933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:37.583638 31927 server_base.cc:1034] running on GCE node
I20260807 19:36:37.585151 31927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:37.586761 31927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:37.588037 31927 hybrid_clock.cc:648] HybridClock initialized: now 1786131397587976 us; error 61 us; skew 500 ppm
I20260807 19:36:37.588516 31927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:37 dist-test-slave-8w9v krb5kdc[31914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:37.592056 31927 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:37.593786 31927 webserver.cc:460] Webserver started at http://127.0.0.1:37753/ using document root <none> and password file <none>
I20260807 19:36:37.594563 31927 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:37.594725 31927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:37.595065 31927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:37.598343 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "5290957f00fb48b89b696f2fbe2d0de4"
format_stamp: "Formatted at 2026-08-07 19:36:37 on dist-test-slave-8w9v"
server_key: "4ea8c739dabcb5cd889af3c0d4a91a0d"
server_key_iv: "5b5b9ad7268c1e79fb0d9715629e7655"
server_key_version: "encryptionkey@0"
I20260807 19:36:37.599170 31927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "5290957f00fb48b89b696f2fbe2d0de4"
format_stamp: "Formatted at 2026-08-07 19:36:37 on dist-test-slave-8w9v"
server_key: "4ea8c739dabcb5cd889af3c0d4a91a0d"
server_key_iv: "5b5b9ad7268c1e79fb0d9715629e7655"
server_key_version: "encryptionkey@0"
I20260807 19:36:37.604851 31927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:36:37.608819 31942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:37.610497 31927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:37.610726 31927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "5290957f00fb48b89b696f2fbe2d0de4"
format_stamp: "Formatted at 2026-08-07 19:36:37 on dist-test-slave-8w9v"
server_key: "4ea8c739dabcb5cd889af3c0d4a91a0d"
server_key_iv: "5b5b9ad7268c1e79fb0d9715629e7655"
server_key_version: "encryptionkey@0"
I20260807 19:36:37.610942 31927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:37.629346 31927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:37.634253 31927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:37.634608 31927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:37.656749 31927 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33075
I20260807 19:36:37.656795 31998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33075 every 8 connection(s)
I20260807 19:36:37.658434 31927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:37.661031 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 31927
I20260807 19:36:37.661258 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:37.661751 9058 external_mini_cluster.cc:1428] Setting key 6482ed13f0969fe7a2b0d9eafe833027
I20260807 19:36:37.664515 32000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:37.681926 32000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4: Bootstrap starting.
I20260807 19:36:37.683470 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:37.663944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:46841 (local address 127.0.0.1:33075)
0807 19:36:37.665196 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:37.665213 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:37.665270 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:36:37.667935 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:37.667979 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:37.668761 (+ 782us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:37.669446 (+ 685us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:37.670303 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:37.673455 (+ 3152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:37.675422 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:37.676956 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:37.677304 (+ 348us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:37.679374 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:37.679397 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:37.679431 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:37.680562 (+ 1131us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:37.680678 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:37.680717 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:37.680909 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:37.681270 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 19:36:37.681487 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":727,"threads_started":1}
I20260807 19:36:37.686025 32000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:37.687464 32000 log.cc:824] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:37.688469 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:37.690819 32000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4: No bootstrap required, opened a new log
I20260807 19:36:37.701393 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:37.693633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54747 (local address 127.0.0.1:33075)
0807 19:36:37.693842 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:37.693854 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:37.694447 (+ 593us) server_negotiation.cc:407] Connection header received
0807 19:36:37.694635 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:37.694641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:37.694744 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:37.694876 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:37.695449 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:37.696749 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:37.698332 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:37.699073 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:37.699210 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:37.700533 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:37.700540 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:37.700544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:37.700571 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:37.700625 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:37.700645 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:37.700764 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:37.700955 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 19:36:37.701011 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:36:37.701125 32000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 19:36:37.701682 32000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:37.701839 32000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5290957f00fb48b89b696f2fbe2d0de4, State: Initialized, Role: FOLLOWER
I20260807 19:36:37.702589 32000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 19:36:37.702907 32000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:37.703066 32000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:37.703266 32000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:37.706432 32000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 19:36:37.706938 32000 leader_election.cc:302] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5290957f00fb48b89b696f2fbe2d0de4; no voters:
I20260807 19:36:37.707511 32000 leader_election.cc:288] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:37.707643 32010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:37.708551 32010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [term 1 LEADER]: Becoming Leader. State: Replica: 5290957f00fb48b89b696f2fbe2d0de4, State: Running, Role: LEADER
I20260807 19:36:37.709157 32010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } }
I20260807 19:36:37.710038 32000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:37.711997 32011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5290957f00fb48b89b696f2fbe2d0de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } } }
I20260807 19:36:37.712495 32011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:37.712586 32013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5290957f00fb48b89b696f2fbe2d0de4. Latest consensus state: current_term: 1 leader_uuid: "5290957f00fb48b89b696f2fbe2d0de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5290957f00fb48b89b696f2fbe2d0de4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33075 } } }
I20260807 19:36:37.714648 32013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:37.714732 32016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:37.720849 32016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:37.730556 32016 catalog_manager.cc:1347] Generated new cluster ID: 744e2efddaff4199a0eeddda602e59c3
I20260807 19:36:37.730697 32016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:37.746510 32016 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:37.749007 32016 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:37.758612 32016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5290957f00fb48b89b696f2fbe2d0de4: Generated new TSK 0
I20260807 19:36:37.759733 32016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:37.779737 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31927
W20260807 19:36:37.781961 31994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:46841
W20260807 19:36:37.783284 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 31927: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:37.783491 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 31927
W20260807 19:36:37.807279 31912 connection.cc:535] client connection to 127.0.0.1:33075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33075): Connection reset by peer (error 104)
2026-08-07T19:36:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:37 dist-test-slave-8w9v krb5kdc[32033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:37 dist-test-slave-8w9v krb5kdc[32033](info): set up 2 sockets
Aug 07 19:36:37 dist-test-slave-8w9v krb5kdc[32033](info): commencing operation
krb5kdc: starting...
W20260807 19:36:38.995433 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.003s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:39 dist-test-slave-8w9v krb5kdc[32033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:39.346174 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:39.704648 32046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:39.704952 32046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:39.705049 32046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:39.712391 32046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:39.712515 32046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:39.712564 32046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:39.712630 32046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:39.712668 32046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:39.722687 32046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:39.723981 32046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:39.725347 32046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:39.737501 32053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:39.737741 32052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:39.737694 32055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:39.738972 32046 server_base.cc:1034] running on GCE node
I20260807 19:36:39.739795 32046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:39.741345 32046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:39.742589 32046 hybrid_clock.cc:648] HybridClock initialized: now 1786131399742538 us; error 47 us; skew 500 ppm
I20260807 19:36:39.742955 32046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:39 dist-test-slave-8w9v krb5kdc[32033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131399, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:39.747594 32046 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:39.749691 32046 webserver.cc:460] Webserver started at http://127.0.0.1:46563/ using document root <none> and password file <none>
I20260807 19:36:39.750557 32046 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:39.750703 32046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:39.751096 32046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:39.755019 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6"
format_stamp: "Formatted at 2026-08-07 19:36:39 on dist-test-slave-8w9v"
server_key: "c3e614c99d6e95a6db50089a3c5286ab"
server_key_iv: "01cfd3697f64f35adc01d1174e968166"
server_key_version: "encryptionkey@0"
I20260807 19:36:39.755841 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6"
format_stamp: "Formatted at 2026-08-07 19:36:39 on dist-test-slave-8w9v"
server_key: "c3e614c99d6e95a6db50089a3c5286ab"
server_key_iv: "01cfd3697f64f35adc01d1174e968166"
server_key_version: "encryptionkey@0"
I20260807 19:36:39.761454 32046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:36:39.765053 32062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:39.766355 32046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:39.766563 32046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6"
format_stamp: "Formatted at 2026-08-07 19:36:39 on dist-test-slave-8w9v"
server_key: "c3e614c99d6e95a6db50089a3c5286ab"
server_key_iv: "01cfd3697f64f35adc01d1174e968166"
server_key_version: "encryptionkey@0"
I20260807 19:36:39.766782 32046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:39.773070 32046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:39.777916 32046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:39.778254 32046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:39.798451 32046 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38589
I20260807 19:36:39.798504 32118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38589 every 8 connection(s)
I20260807 19:36:39.800077 32046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:39.803937 32119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:39.810204 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32046
I20260807 19:36:39.810407 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:39.810838 9058 external_mini_cluster.cc:1428] Setting key e9cc3ee3b744bf8cf17a22b01678ac81
I20260807 19:36:39.816529 32119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6: Bootstrap starting.
I20260807 19:36:39.820380 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:39.813397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:33061 (local address 127.0.0.1:38589)
0807 19:36:39.814400 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:39.814414 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:39.814462 (+ 48us) server_negotiation.cc:407] Connection header received
0807 19:36:39.816629 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:39.816655 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:39.817080 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:39.817430 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:39.818691 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:39.818697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:39.818710 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:39.818724 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:39.818791 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:39.818813 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:39.818998 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:39.819110 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:36:39.819331 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":427,"threads_started":1}
I20260807 19:36:39.820441 32119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:39.821908 32119 log.cc:824] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:39.824883 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:39.825599 32119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6: No bootstrap required, opened a new log
I20260807 19:36:39.832240 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:39.829399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:38805 (local address 127.0.0.1:38589)
0807 19:36:39.829596 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:39.829603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:39.830087 (+ 484us) server_negotiation.cc:407] Connection header received
0807 19:36:39.830325 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:39.830331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:39.830405 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:39.830521 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:39.831501 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:39.831506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:39.831511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:39.831530 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:39.831584 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:39.831600 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:39.831745 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:39.831820 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 19:36:39.831885 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:36:39.834692 32119 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } }
I20260807 19:36:39.835124 32119 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:39.835248 32119 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bd5055e3bd4e8b9950eb2625b8d7e6, State: Initialized, Role: FOLLOWER
I20260807 19:36:39.835846 32119 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } }
I20260807 19:36:39.836127 32119 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:39.836349 32119 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:39.836536 32119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:39.838917 32119 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } }
I20260807 19:36:39.839447 32119 leader_election.cc:302] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bd5055e3bd4e8b9950eb2625b8d7e6; no voters:
I20260807 19:36:39.840415 32119 leader_election.cc:288] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:39.840494 32129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:39.841270 32129 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [term 1 LEADER]: Becoming Leader. State: Replica: f9bd5055e3bd4e8b9950eb2625b8d7e6, State: Running, Role: LEADER
I20260807 19:36:39.841856 32129 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } }
I20260807 19:36:39.842682 32119 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:39.844936 32131 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bd5055e3bd4e8b9950eb2625b8d7e6. Latest consensus state: current_term: 1 leader_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } } }
I20260807 19:36:39.845499 32131 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:39.850229 32130 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd5055e3bd4e8b9950eb2625b8d7e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38589 } } }
I20260807 19:36:39.850520 32130 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:39.851244 32135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:39.856784 32135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:39.865597 32135 catalog_manager.cc:1347] Generated new cluster ID: 6131997062a946bb814e6bfd52d6ca3f
I20260807 19:36:39.865764 32135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:39.882401 32135 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:39.882958 32135 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:39.892243 32135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9bd5055e3bd4e8b9950eb2625b8d7e6: Generated new TSK 0
I20260807 19:36:39.893388 32135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:39.903105 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32046
W20260807 19:36:39.904780 32114 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:33061
W20260807 19:36:39.905543 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32046: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:39.905697 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32046
2026-08-07T19:36:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2118 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:40 dist-test-slave-8w9v krb5kdc[32150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:40 dist-test-slave-8w9v krb5kdc[32150](info): set up 2 sockets
Aug 07 19:36:40 dist-test-slave-8w9v krb5kdc[32150](info): commencing operation
krb5kdc: starting...
W20260807 19:36:41.094727 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:41 dist-test-slave-8w9v krb5kdc[32150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:41.443420 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:41.803548 32164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:41.803895 32164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:41.804001 32164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:41.811945 32164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:41.812077 32164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:41.812175 32164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:41.812263 32164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:41.812312 32164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:41.824311 32164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:41.825718 32164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:41.827029 32164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:41.839452 32171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:41.839541 32173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:41.839516 32170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:41.840879 32164 server_base.cc:1034] running on GCE node
I20260807 19:36:41.841740 32164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:41.843174 32164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:41.844376 32164 hybrid_clock.cc:648] HybridClock initialized: now 1786131401844318 us; error 54 us; skew 500 ppm
I20260807 19:36:41.844749 32164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:41 dist-test-slave-8w9v krb5kdc[32150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131401, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:41.848389 32164 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:41.849962 32164 webserver.cc:460] Webserver started at http://127.0.0.1:34827/ using document root <none> and password file <none>
I20260807 19:36:41.850694 32164 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:41.850850 32164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:41.851179 32164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:41.854380 32164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "3bd5b8bb52da49cea7fb216c29a0073d"
format_stamp: "Formatted at 2026-08-07 19:36:41 on dist-test-slave-8w9v"
server_key: "7ff2b0213108cb8662289ea2ed252014"
server_key_iv: "4287c20e8b7b82d97a4ac4155b6aa61b"
server_key_version: "encryptionkey@0"
I20260807 19:36:41.855180 32164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "3bd5b8bb52da49cea7fb216c29a0073d"
format_stamp: "Formatted at 2026-08-07 19:36:41 on dist-test-slave-8w9v"
server_key: "7ff2b0213108cb8662289ea2ed252014"
server_key_iv: "4287c20e8b7b82d97a4ac4155b6aa61b"
server_key_version: "encryptionkey@0"
I20260807 19:36:41.860795 32164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:36:41.864480 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:41.865943 32164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:36:41.866147 32164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "3bd5b8bb52da49cea7fb216c29a0073d"
format_stamp: "Formatted at 2026-08-07 19:36:41 on dist-test-slave-8w9v"
server_key: "7ff2b0213108cb8662289ea2ed252014"
server_key_iv: "4287c20e8b7b82d97a4ac4155b6aa61b"
server_key_version: "encryptionkey@0"
I20260807 19:36:41.866331 32164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:41.910614 32164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:41.914990 32164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:41.915251 32164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:41.935148 32164 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44855
I20260807 19:36:41.935187 32233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44855 every 8 connection(s)
I20260807 19:36:41.936642 32164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:41.937669 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32164
I20260807 19:36:41.937872 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:41.938452 9058 external_mini_cluster.cc:1428] Setting key 55d89a0b1b22e1ac4802b488c70f0a3e
I20260807 19:36:41.941327 32234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:36:41 dist-test-slave-8w9v krb5kdc[32150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:36:41.955240 32234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d: Bootstrap starting.
I20260807 19:36:41.959759 32234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:41.961252 32234 log.cc:824] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:41.964085 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:41.940394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:33317 (local address 127.0.0.1:44855)
0807 19:36:41.941176 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:41.941188 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:41.941229 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:36:41.942903 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:41.942925 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:41.943292 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:41.943609 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:41.945450 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:41.946795 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:41.948734 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:41.949401 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:41.949541 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:41.954674 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:41.954684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:41.954707 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:41.954764 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:41.958914 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:41.959888 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:41.959902 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:41.959913 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:41.960193 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:41.960700 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:41.960705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:41.960708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:41.961491 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:41.961849 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:41.962295 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 19:36:41.962667 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":355,"threads_started":1}
I20260807 19:36:41.965020 32234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d: No bootstrap required, opened a new log
I20260807 19:36:41.968180 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:41.973851 32234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } }
I20260807 19:36:41.974365 32234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:41.974501 32234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd5b8bb52da49cea7fb216c29a0073d, State: Initialized, Role: FOLLOWER
I20260807 19:36:41.975119 32234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } }
I20260807 19:36:41.975358 32234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:41.975548 32234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:41.975723 32234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:41.978816 32234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } }
I20260807 19:36:41.979455 32234 leader_election.cc:302] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd5b8bb52da49cea7fb216c29a0073d; no voters:
I20260807 19:36:41.980165 32234 leader_election.cc:288] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:41.980329 32246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:41.981756 32246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [term 1 LEADER]: Becoming Leader. State: Replica: 3bd5b8bb52da49cea7fb216c29a0073d, State: Running, Role: LEADER
I20260807 19:36:41.982617 32234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:41.982486 32246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } }
I20260807 19:36:41.986378 32248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd5b8bb52da49cea7fb216c29a0073d. Latest consensus state: current_term: 1 leader_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } } }
I20260807 19:36:41.986865 32248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:41.987703 32247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd5b8bb52da49cea7fb216c29a0073d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44855 } } }
I20260807 19:36:41.989064 32247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:41.990319 32255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:41.994102 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:41.973398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:48629 (local address 127.0.0.1:44855)
0807 19:36:41.973602 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:41.973608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:41.974160 (+ 552us) server_negotiation.cc:407] Connection header received
0807 19:36:41.974298 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:41.974303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:41.974402 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:41.974543 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:41.976675 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:41.977858 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:41.979297 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:41.980047 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:41.980271 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:41.983944 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:41.983953 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:41.983962 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:41.984006 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:41.989919 (+ 5913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:41.990941 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:41.990949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:41.990958 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:41.991159 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:41.991659 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:41.991667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:41.991672 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:41.992203 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:41.992567 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:41.993520 (+ 953us) server_negotiation.cc:299] Negotiation successful
0807 19:36:41.993729 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:36:41.997330 32255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:42.008596 32255 catalog_manager.cc:1347] Generated new cluster ID: 83c9ad4284c24713a3f8a2dba918ad7a
I20260807 19:36:42.008745 32255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:42.024214 32255 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:42.025566 32255 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:42.036029 32255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bd5b8bb52da49cea7fb216c29a0073d: Generated new TSK 0
I20260807 19:36:42.036983 32255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:42.047104 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32164
I20260807 19:36:42.048384 32229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:42.142627 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32164
W20260807 19:36:42.162076 32148 connection.cc:535] client connection to 127.0.0.1:44855 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44855): Connection reset by peer (error 104)
2026-08-07T19:36:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2240 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:42 dist-test-slave-8w9v krb5kdc[32268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:42 dist-test-slave-8w9v krb5kdc[32268](info): set up 2 sockets
Aug 07 19:36:42 dist-test-slave-8w9v krb5kdc[32268](info): commencing operation
krb5kdc: starting...
W20260807 19:36:43.359346 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:43 dist-test-slave-8w9v krb5kdc[32268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:43.701690 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:44.073982 32281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:44.074327 32281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:44.074505 32281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:44.082417 32281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:44.082568 32281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:44.082616 32281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:44.082674 32281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:44.082713 32281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:44.093744 32281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:44.095293 32281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:44.096724 32281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:44.109702 32288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:44.109644 32291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:44.109727 32287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:44.110576 32281 server_base.cc:1034] running on GCE node
I20260807 19:36:44.111359 32281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:44.112960 32281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:44.114203 32281 hybrid_clock.cc:648] HybridClock initialized: now 1786131404114162 us; error 27 us; skew 500 ppm
I20260807 19:36:44.114595 32281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:44 dist-test-slave-8w9v krb5kdc[32268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131404, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:44.122975 32281 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:44.124820 32281 webserver.cc:460] Webserver started at http://127.0.0.1:41869/ using document root <none> and password file <none>
I20260807 19:36:44.125564 32281 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:44.125679 32281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:44.125958 32281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:44.129292 32281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "4e739b56754d4d9c96e1e6e61b1f72e9"
format_stamp: "Formatted at 2026-08-07 19:36:44 on dist-test-slave-8w9v"
server_key: "69abd2bad618fcaf1dfa0fe9184711d6"
server_key_iv: "7a609441abbfc9fe38cd4b5c1cd8717e"
server_key_version: "encryptionkey@0"
I20260807 19:36:44.130071 32281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "4e739b56754d4d9c96e1e6e61b1f72e9"
format_stamp: "Formatted at 2026-08-07 19:36:44 on dist-test-slave-8w9v"
server_key: "69abd2bad618fcaf1dfa0fe9184711d6"
server_key_iv: "7a609441abbfc9fe38cd4b5c1cd8717e"
server_key_version: "encryptionkey@0"
I20260807 19:36:44.135653 32281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:36:44.139448 32300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:44.140996 32281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:44.141211 32281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "4e739b56754d4d9c96e1e6e61b1f72e9"
format_stamp: "Formatted at 2026-08-07 19:36:44 on dist-test-slave-8w9v"
server_key: "69abd2bad618fcaf1dfa0fe9184711d6"
server_key_iv: "7a609441abbfc9fe38cd4b5c1cd8717e"
server_key_version: "encryptionkey@0"
I20260807 19:36:44.141429 32281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:44.160038 32281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:44.164551 32281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:44.164896 32281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:44.185173 32281 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38609
I20260807 19:36:44.185210 32354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38609 every 8 connection(s)
I20260807 19:36:44.186760 32281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:44.190639 32355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:44.196954 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32281
I20260807 19:36:44.197170 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:44.197685 9058 external_mini_cluster.cc:1428] Setting key 4381f890fc32d68537d025c3326d3bfc
I20260807 19:36:44.201577 32355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9: Bootstrap starting.
I20260807 19:36:44.205075 32355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:44.206391 32355 log.cc:824] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:44.209868 32355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9: No bootstrap required, opened a new log
I20260807 19:36:44.212253 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:44.199742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:43157 (local address 127.0.0.1:38609)
0807 19:36:44.200648 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:44.200660 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:44.200704 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:36:44.202516 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:44.202538 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:44.202945 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:44.203293 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:44.203931 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:44.205426 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:44.207346 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:44.208293 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:44.208450 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:44.209740 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:44.209747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:44.209760 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:44.210387 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:44.210474 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:44.210501 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:44.210613 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:44.210933 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 19:36:44.211142 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":434,"threads_started":1}
I20260807 19:36:44.216332 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:44.218299 32355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 19:36:44.218698 32355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:44.218837 32355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e739b56754d4d9c96e1e6e61b1f72e9, State: Initialized, Role: FOLLOWER
I20260807 19:36:44.219467 32355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 19:36:44.219713 32355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:44.219877 32355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:44.220055 32355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:44.222854 32355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 19:36:44.223424 32355 leader_election.cc:302] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e739b56754d4d9c96e1e6e61b1f72e9; no voters:
I20260807 19:36:44.224200 32355 leader_election.cc:288] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:44.224254 32366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:44.227031 32355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:44.228046 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:44.220971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:42999 (local address 127.0.0.1:38609)
0807 19:36:44.221165 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:44.221172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:44.221557 (+ 385us) server_negotiation.cc:407] Connection header received
0807 19:36:44.221759 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:44.221764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:44.221852 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:44.221968 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:44.222522 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:44.223386 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:44.225089 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:44.225742 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:44.225894 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:44.227132 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:44.227139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:44.227143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:44.227167 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:44.227241 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:44.227271 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:44.227397 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:44.227618 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:36:44.227678 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:36:44.231055 32366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [term 1 LEADER]: Becoming Leader. State: Replica: 4e739b56754d4d9c96e1e6e61b1f72e9, State: Running, Role: LEADER
I20260807 19:36:44.231777 32366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } }
I20260807 19:36:44.235509 32367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e739b56754d4d9c96e1e6e61b1f72e9. Latest consensus state: current_term: 1 leader_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } } }
I20260807 19:36:44.236038 32367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:44.235534 32368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e739b56754d4d9c96e1e6e61b1f72e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38609 } } }
I20260807 19:36:44.236544 32368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:44.243541 32385 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:36:44.244308 32377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:44.249634 32377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:44.258535 32377 catalog_manager.cc:1347] Generated new cluster ID: f128602e46684a4c998e413ca7ef2fd0
I20260807 19:36:44.258710 32377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:44.273123 32377 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:44.274827 32377 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:44.296012 32377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e739b56754d4d9c96e1e6e61b1f72e9: Generated new TSK 0
I20260807 19:36:44.297062 32377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:44.312345 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32281
W20260807 19:36:44.314695 32350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:43157
W20260807 19:36:44.315552 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32281: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:44.315717 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32281
W20260807 19:36:44.337802 32266 connection.cc:535] client connection to 127.0.0.1:38609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38609): Connection reset by peer (error 104)
2026-08-07T19:36:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:44 dist-test-slave-8w9v krb5kdc[32391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:44 dist-test-slave-8w9v krb5kdc[32391](info): set up 2 sockets
Aug 07 19:36:44 dist-test-slave-8w9v krb5kdc[32391](info): commencing operation
krb5kdc: starting...
W20260807 19:36:45.531734 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:45 dist-test-slave-8w9v krb5kdc[32391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:36:45.883386 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:46.250777 32404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:46.251118 32404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:46.251320 32404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:46.259018 32404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:46.259145 32404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:46.259270 32404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:46.259356 32404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:46.259429 32404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:46.270044 32404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:46.271493 32404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:46.272945 32404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:46.286609 32412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:46.286728 32410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:46.286559 32409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:46.289659 32404 server_base.cc:1034] running on GCE node
I20260807 19:36:46.290531 32404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:46.292054 32404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:46.293323 32404 hybrid_clock.cc:648] HybridClock initialized: now 1786131406293254 us; error 65 us; skew 500 ppm
I20260807 19:36:46.293705 32404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:46 dist-test-slave-8w9v krb5kdc[32391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:46.297514 32404 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:36:46.299324 32404 webserver.cc:460] Webserver started at http://127.0.0.1:33135/ using document root <none> and password file <none>
I20260807 19:36:46.300037 32404 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:46.300209 32404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:46.300510 32404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:46.303567 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "c5895bb2824f4d0caa5991f636e21985"
format_stamp: "Formatted at 2026-08-07 19:36:46 on dist-test-slave-8w9v"
server_key: "8a5b5ed99934efde4e9495ea515957af"
server_key_iv: "0f5f76f7f5fb36b98787eb0958240358"
server_key_version: "encryptionkey@0"
I20260807 19:36:46.304396 32404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "c5895bb2824f4d0caa5991f636e21985"
format_stamp: "Formatted at 2026-08-07 19:36:46 on dist-test-slave-8w9v"
server_key: "8a5b5ed99934efde4e9495ea515957af"
server_key_iv: "0f5f76f7f5fb36b98787eb0958240358"
server_key_version: "encryptionkey@0"
I20260807 19:36:46.309713 32404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:36:46.313345 32419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:46.314803 32404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:36:46.315017 32404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "c5895bb2824f4d0caa5991f636e21985"
format_stamp: "Formatted at 2026-08-07 19:36:46 on dist-test-slave-8w9v"
server_key: "8a5b5ed99934efde4e9495ea515957af"
server_key_iv: "0f5f76f7f5fb36b98787eb0958240358"
server_key_version: "encryptionkey@0"
I20260807 19:36:46.315217 32404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:46.321444 32404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:46.326233 32404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:46.326568 32404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:46.346057 32404 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39981
I20260807 19:36:46.346112 32474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39981 every 8 connection(s)
I20260807 19:36:46.347553 32404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:46.348951 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32404
I20260807 19:36:46.349160 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:46.349754 9058 external_mini_cluster.cc:1428] Setting key a07174f3b31ec5f464bebfc07b737d85
I20260807 19:36:46.352411 32475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:46.357955 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:46.351657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:59237 (local address 127.0.0.1:39981)
0807 19:36:46.352585 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:46.352598 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:46.352640 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:36:46.354358 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:46.354381 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:46.354733 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:46.355088 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:46.356250 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:46.356256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:46.356268 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:46.356328 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:46.356423 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:46.356447 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:46.356629 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:46.356733 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 19:36:46.356984 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":406,"threads_started":1}
I20260807 19:36:46.361665 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:46.364085 32475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985: Bootstrap starting.
I20260807 19:36:46.367925 32475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:46.368945 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:46.366351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54363 (local address 127.0.0.1:39981)
0807 19:36:46.366550 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:46.366556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:46.366808 (+ 252us) server_negotiation.cc:407] Connection header received
0807 19:36:46.367005 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:46.367009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:46.367058 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:46.367140 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:46.368202 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:46.368206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:46.368209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:46.368225 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:46.368266 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:46.368282 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:46.368382 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:46.368558 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 19:36:46.368613 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:36:46.369369 32475 log.cc:824] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:46.372383 32475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985: No bootstrap required, opened a new log
I20260807 19:36:46.379845 32475 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } }
I20260807 19:36:46.380336 32475 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:46.380456 32475 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5895bb2824f4d0caa5991f636e21985, State: Initialized, Role: FOLLOWER
I20260807 19:36:46.381048 32475 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } }
I20260807 19:36:46.381295 32475 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:46.381457 32475 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:46.381639 32475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:46.383889 32475 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } }
I20260807 19:36:46.384472 32475 leader_election.cc:302] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5895bb2824f4d0caa5991f636e21985; no voters:
I20260807 19:36:46.385226 32475 leader_election.cc:288] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:46.385349 32486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:46.386021 32486 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [term 1 LEADER]: Becoming Leader. State: Replica: c5895bb2824f4d0caa5991f636e21985, State: Running, Role: LEADER
I20260807 19:36:46.386683 32486 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } }
I20260807 19:36:46.387720 32475 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:46.390019 32489 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5895bb2824f4d0caa5991f636e21985. Latest consensus state: current_term: 1 leader_uuid: "c5895bb2824f4d0caa5991f636e21985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } } }
I20260807 19:36:46.390225 32487 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5895bb2824f4d0caa5991f636e21985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5895bb2824f4d0caa5991f636e21985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39981 } } }
I20260807 19:36:46.390558 32489 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:46.390591 32487 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:46.391561 32495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:46.398331 32495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:46.407730 32495 catalog_manager.cc:1347] Generated new cluster ID: 340f0a332e3b488ca3543ead99f37959
I20260807 19:36:46.407876 32495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:46.424759 32495 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:46.425220 32495 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:46.434947 32495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5895bb2824f4d0caa5991f636e21985: Generated new TSK 0
I20260807 19:36:46.436156 32495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:46.451156 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32404
W20260807 19:36:46.452908 32470 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:59237
W20260807 19:36:46.453647 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32404: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:46.453802 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32404
2026-08-07T19:36:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2137 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:46 dist-test-slave-8w9v krb5kdc[32510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:46 dist-test-slave-8w9v krb5kdc[32510](info): set up 2 sockets
Aug 07 19:36:46 dist-test-slave-8w9v krb5kdc[32510](info): commencing operation
krb5kdc: starting...
W20260807 19:36:47.663159 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:47 dist-test-slave-8w9v krb5kdc[32510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:48.024684 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36029
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:32967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:36029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:48.388899 32523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:48.389204 32523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:48.389390 32523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:48.396952 32523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:48.397082 32523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:48.397183 32523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:48.397302 32523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:48.397372 32523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:48.407837 32523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:32967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:36029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:36029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:48.409364 32523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:48.410804 32523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:48.423079 32529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:48.423177 32531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:48.423115 32528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:48.424104 32523 server_base.cc:1034] running on GCE node
I20260807 19:36:48.425073 32523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:48.426743 32523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:48.427985 32523 hybrid_clock.cc:648] HybridClock initialized: now 1786131408427940 us; error 94 us; skew 500 ppm
I20260807 19:36:48.428423 32523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:48 dist-test-slave-8w9v krb5kdc[32510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131408, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:48.431874 32523 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:48.433550 32523 webserver.cc:460] Webserver started at http://127.8.216.190:42477/ using document root <none> and password file <none>
I20260807 19:36:48.434234 32523 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:48.434345 32523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:48.434617 32523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:48.437812 32523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "75e7f69684e949a28ee0c33c49cb365b"
format_stamp: "Formatted at 2026-08-07 19:36:48 on dist-test-slave-8w9v"
server_key: "96495b596b460cd5adb645529e7b7dd8"
server_key_iv: "4a6f25ca108f17501751273f965ef330"
server_key_version: "encryptionkey@0"
I20260807 19:36:48.438585 32523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "75e7f69684e949a28ee0c33c49cb365b"
format_stamp: "Formatted at 2026-08-07 19:36:48 on dist-test-slave-8w9v"
server_key: "96495b596b460cd5adb645529e7b7dd8"
server_key_iv: "4a6f25ca108f17501751273f965ef330"
server_key_version: "encryptionkey@0"
I20260807 19:36:48.444358 32523 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 19:36:48.448261 32539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:48.449820 32523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:36:48.450031 32523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "75e7f69684e949a28ee0c33c49cb365b"
format_stamp: "Formatted at 2026-08-07 19:36:48 on dist-test-slave-8w9v"
server_key: "96495b596b460cd5adb645529e7b7dd8"
server_key_iv: "4a6f25ca108f17501751273f965ef330"
server_key_version: "encryptionkey@0"
I20260807 19:36:48.450273 32523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:48.469862 32523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:48.474512 32523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:48.474854 32523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:48.495597 32523 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:36029
I20260807 19:36:48.495618 32591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:36029 every 8 connection(s)
I20260807 19:36:48.497146 32523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:48.500558 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32523
I20260807 19:36:48.500761 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:48.501271 9058 external_mini_cluster.cc:1428] Setting key bc637173416c26ff879c6f78b45157f2
I20260807 19:36:48.501415 32593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:48.516932 32593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b: Bootstrap starting.
Aug 07 19:36:48 dist-test-slave-8w9v krb5kdc[32510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:48.521387 32593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:48.522910 32593 log.cc:824] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:48.526108 32593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b: No bootstrap required, opened a new log
I20260807 19:36:48.527591 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:48.503224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53887 (local address 127.8.216.190:36029)
0807 19:36:48.504052 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:48.504068 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:48.504139 (+ 71us) server_negotiation.cc:407] Connection header received
0807 19:36:48.506576 (+ 2437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:48.506609 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:48.507193 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:48.507704 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:48.509259 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:48.511309 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:48.512770 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:48.513937 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:48.514133 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:48.518580 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:48.518594 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:48.518621 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:48.518690 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:48.523014 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:48.523934 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:48.523942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:48.523948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:48.524221 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:48.524752 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:48.524764 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:48.524767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:48.525317 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:48.525666 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:48.526210 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 19:36:48.526572 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":340,"threads_started":1}
I20260807 19:36:48.532115 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:48.535300 32593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } }
I20260807 19:36:48.535738 32593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:48.535877 32593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75e7f69684e949a28ee0c33c49cb365b, State: Initialized, Role: FOLLOWER
I20260807 19:36:48.536643 32593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } }
I20260807 19:36:48.537001 32593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:48.537158 32593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:48.537392 32593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:48.540378 32593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } }
I20260807 19:36:48.541010 32593 leader_election.cc:302] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75e7f69684e949a28ee0c33c49cb365b; no voters:
I20260807 19:36:48.541695 32593 leader_election.cc:288] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:48.541857 32606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:48.543956 32593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:48.544414 32606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [term 1 LEADER]: Becoming Leader. State: Replica: 75e7f69684e949a28ee0c33c49cb365b, State: Running, Role: LEADER
I20260807 19:36:48.545181 32606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } }
I20260807 19:36:48.553665 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75e7f69684e949a28ee0c33c49cb365b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } } }
I20260807 19:36:48.554103 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:48.554793 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75e7f69684e949a28ee0c33c49cb365b. Latest consensus state: current_term: 1 leader_uuid: "75e7f69684e949a28ee0c33c49cb365b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e7f69684e949a28ee0c33c49cb365b" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 36029 } } }
I20260807 19:36:48.555210 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:48.560353 32598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:48.536973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57967 (local address 127.8.216.190:36029)
0807 19:36:48.537197 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:48.537205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:48.537660 (+ 455us) server_negotiation.cc:407] Connection header received
0807 19:36:48.537804 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:48.537811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:48.537901 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:48.538019 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:48.539488 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:48.540647 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:48.542332 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:48.543324 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:48.543470 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:48.547598 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:48.547606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:48.547611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:48.547655 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:48.553367 (+ 5712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:48.555066 (+ 1699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:48.555075 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:48.555085 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:48.555284 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:48.556172 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:48.556180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:48.556185 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:48.556802 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:48.559036 (+ 2234us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:48.559365 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 19:36:48.559675 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:36:48.569013 32617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:48.575786 32617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:48.586109 32617 catalog_manager.cc:1347] Generated new cluster ID: 5f9c3c9220484a40a3a5cdc6ca0e0750
I20260807 19:36:48.586318 32617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:48.594605 32617 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:48.596568 32617 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:48.610409 32617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75e7f69684e949a28ee0c33c49cb365b: Generated new TSK 0
I20260807 19:36:48.611526 32617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:48.623695 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32523
I20260807 19:36:48.625085 32587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:48.718708 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32523
W20260807 19:36:48.740084 32508 connection.cc:535] client connection to 127.8.216.190:36029 recv error: Network error: failed to read from TLS socket (remote: 127.8.216.190:36029): Connection reset by peer (error 104)
2026-08-07T19:36:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:48 dist-test-slave-8w9v krb5kdc[32629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:48 dist-test-slave-8w9v krb5kdc[32629](info): set up 2 sockets
Aug 07 19:36:48 dist-test-slave-8w9v krb5kdc[32629](info): commencing operation
krb5kdc: starting...
W20260807 19:36:49.908974 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:50 dist-test-slave-8w9v krb5kdc[32629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:50.273547 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42449
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:42449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:50.635685 32642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:50.636046 32642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:50.636298 32642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:50.643886 32642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:50.644068 32642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:50.644205 32642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:50.644318 32642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:50.644413 32642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:50.655007 32642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:42449
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:50.656555 32642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:50.657956 32642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:50.670399 32652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:50.670368 32650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:50.670353 32649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:50.673671 32642 server_base.cc:1034] running on GCE node
I20260807 19:36:50.674515 32642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:50.676065 32642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:50.677330 32642 hybrid_clock.cc:648] HybridClock initialized: now 1786131410677290 us; error 28 us; skew 500 ppm
I20260807 19:36:50.677717 32642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:50 dist-test-slave-8w9v krb5kdc[32629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:50.686019 32642 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:50.687726 32642 webserver.cc:460] Webserver started at http://127.8.216.190:43135/ using document root <none> and password file <none>
I20260807 19:36:50.688607 32642 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:50.688750 32642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:50.689087 32642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:50.692255 32642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "6937de080dc04b8083cf4f5697cff1bc"
format_stamp: "Formatted at 2026-08-07 19:36:50 on dist-test-slave-8w9v"
server_key: "c66112cd8e712f517cfca92b48b55ba5"
server_key_iv: "7389ce39fcf46963fcda3da2cd1ef76f"
server_key_version: "encryptionkey@0"
I20260807 19:36:50.693068 32642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "6937de080dc04b8083cf4f5697cff1bc"
format_stamp: "Formatted at 2026-08-07 19:36:50 on dist-test-slave-8w9v"
server_key: "c66112cd8e712f517cfca92b48b55ba5"
server_key_iv: "7389ce39fcf46963fcda3da2cd1ef76f"
server_key_version: "encryptionkey@0"
I20260807 19:36:50.698565 32642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:36:50.702309 32659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:50.703758 32642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:36:50.703972 32642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "6937de080dc04b8083cf4f5697cff1bc"
format_stamp: "Formatted at 2026-08-07 19:36:50 on dist-test-slave-8w9v"
server_key: "c66112cd8e712f517cfca92b48b55ba5"
server_key_iv: "7389ce39fcf46963fcda3da2cd1ef76f"
server_key_version: "encryptionkey@0"
I20260807 19:36:50.704231 32642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:50.727547 32642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:50.732710 32642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:50.733071 32642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:50.753116 32642 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:42449
I20260807 19:36:50.753140 32713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:42449 every 8 connection(s)
I20260807 19:36:50.754645 32642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:50.758785 32714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:50.760223 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32642
I20260807 19:36:50.760462 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:50.760936 9058 external_mini_cluster.cc:1428] Setting key ec4b38e7a45b057b56d68301629f718f
I20260807 19:36:50.773643 32714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc: Bootstrap starting.
I20260807 19:36:50.775187 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:50.763046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35009 (local address 127.8.216.190:42449)
0807 19:36:50.763940 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:50.763952 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:50.763995 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:36:50.765730 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:50.765756 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:50.766187 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:50.766526 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:50.767162 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:50.768550 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:50.770550 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:50.771277 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:50.771404 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:50.772828 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:50.772835 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:50.772849 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:50.773381 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:50.773469 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:50.773495 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:50.773623 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:50.774005 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 19:36:50.774204 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":434,"threads_started":1}
I20260807 19:36:50.777452 32714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:50.778774 32714 log.cc:824] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:50.779778 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:50.781924 32714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc: No bootstrap required, opened a new log
I20260807 19:36:50.791924 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:50.784680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52479 (local address 127.8.216.190:42449)
0807 19:36:50.784962 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:50.784972 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:50.785528 (+ 556us) server_negotiation.cc:407] Connection header received
0807 19:36:50.785680 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:50.785686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:50.785771 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:50.785893 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:50.786427 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:50.787647 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:50.789063 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:50.789804 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:50.789943 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:50.791102 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:50.791109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:50.791113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:50.791134 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:50.791203 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:50.791226 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:50.791352 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:50.791521 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 19:36:50.791588 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:36:50.791584 32714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } }
I20260807 19:36:50.792196 32714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:50.792337 32714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6937de080dc04b8083cf4f5697cff1bc, State: Initialized, Role: FOLLOWER
I20260807 19:36:50.792985 32714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } }
I20260807 19:36:50.793234 32714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:50.793411 32714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:50.793637 32714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:50.796041 32714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } }
I20260807 19:36:50.796663 32714 leader_election.cc:302] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6937de080dc04b8083cf4f5697cff1bc; no voters:
I20260807 19:36:50.797334 32714 leader_election.cc:288] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:50.797523 32724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:50.798873 32724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [term 1 LEADER]: Becoming Leader. State: Replica: 6937de080dc04b8083cf4f5697cff1bc, State: Running, Role: LEADER
I20260807 19:36:50.799440 32724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } }
I20260807 19:36:50.799842 32714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:50.802273 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6937de080dc04b8083cf4f5697cff1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } } }
I20260807 19:36:50.802652 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:50.803601 32729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:50.803442 32726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6937de080dc04b8083cf4f5697cff1bc. Latest consensus state: current_term: 1 leader_uuid: "6937de080dc04b8083cf4f5697cff1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6937de080dc04b8083cf4f5697cff1bc" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42449 } } }
I20260807 19:36:50.805163 32726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:50.811219 32729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:50.821470 32729 catalog_manager.cc:1347] Generated new cluster ID: f9f3da4d4c104b31bf760b45e9071c66
I20260807 19:36:50.821621 32729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:50.838994 32729 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:50.840435 32729 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:50.847034 32729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6937de080dc04b8083cf4f5697cff1bc: Generated new TSK 0
I20260807 19:36:50.847891 32729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:50.861078 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32642
W20260807 19:36:50.863245 32709 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35009
W20260807 19:36:50.864144 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32642: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:50.864318 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32642
W20260807 19:36:50.886464 32627 connection.cc:535] client connection to 127.8.216.190:42449 recv error: Network error: failed to read from TLS socket (remote: 127.8.216.190:42449): Connection reset by peer (error 104)
2026-08-07T19:36:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2146 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:50 dist-test-slave-8w9v krb5kdc[32749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:50 dist-test-slave-8w9v krb5kdc[32749](info): set up 2 sockets
Aug 07 19:36:50 dist-test-slave-8w9v krb5kdc[32749](info): commencing operation
krb5kdc: starting...
W20260807 19:36:52.043270 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:52 dist-test-slave-8w9v krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:52.395671 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33507
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:40185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:52.766691 32765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:52.767040 32765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:52.767141 32765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:52.774818 32765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:52.774963 32765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:52.775027 32765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:52.775094 32765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:52.775142 32765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:52.785844 32765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:40185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:33507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:52.787191 32765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:52.788599 32765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:52.801635 306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:52.801677 303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:52.801613 304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:52.802937 32765 server_base.cc:1034] running on GCE node
I20260807 19:36:52.803860 32765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:52.805505 32765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:52.806831 32765 hybrid_clock.cc:648] HybridClock initialized: now 1786131412806762 us; error 48 us; skew 500 ppm
I20260807 19:36:52.807255 32765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:52 dist-test-slave-8w9v krb5kdc[32749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:52.810611 32765 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:52.812403 32765 webserver.cc:460] Webserver started at http://127.8.216.190:33555/ using document root <none> and password file <none>
I20260807 19:36:52.813079 32765 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:52.813184 32765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:52.813447 32765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:52.816715 32765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "01f6c784a3c84c2f915fe2c2b0f66368"
format_stamp: "Formatted at 2026-08-07 19:36:52 on dist-test-slave-8w9v"
server_key: "cc9131b3082f96849fa1a77f3fbfdf7c"
server_key_iv: "47f60bf4b03e903d04351d1bd8d07e2b"
server_key_version: "encryptionkey@0"
I20260807 19:36:52.817492 32765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "01f6c784a3c84c2f915fe2c2b0f66368"
format_stamp: "Formatted at 2026-08-07 19:36:52 on dist-test-slave-8w9v"
server_key: "cc9131b3082f96849fa1a77f3fbfdf7c"
server_key_iv: "47f60bf4b03e903d04351d1bd8d07e2b"
server_key_version: "encryptionkey@0"
I20260807 19:36:52.823367 32765 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:36:52.827090 313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:52.828567 32765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:52.828761 32765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "01f6c784a3c84c2f915fe2c2b0f66368"
format_stamp: "Formatted at 2026-08-07 19:36:52 on dist-test-slave-8w9v"
server_key: "cc9131b3082f96849fa1a77f3fbfdf7c"
server_key_iv: "47f60bf4b03e903d04351d1bd8d07e2b"
server_key_version: "encryptionkey@0"
I20260807 19:36:52.828985 32765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:52.835084 32765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:52.839429 32765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:52.839740 32765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:52.859490 32765 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:33507
I20260807 19:36:52.859527 369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:33507 every 8 connection(s)
I20260807 19:36:52.861045 32765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:52.861586 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 32765
I20260807 19:36:52.861793 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:52.862463 9058 external_mini_cluster.cc:1428] Setting key e6bb1b992205bcaeb58b8d551595f556
I20260807 19:36:52.866489 370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:52.873481 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:52.864608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38269 (local address 127.8.216.190:33507)
0807 19:36:52.867900 (+ 3292us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:52.867915 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:52.867959 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:36:52.869702 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:52.869725 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:52.870102 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:52.870481 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:52.871685 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:52.871691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:52.871703 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:52.871718 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:52.871799 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:52.871823 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:52.872010 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:52.872231 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:36:52.872470 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3026,"thread_start_us":2764,"threads_started":1}
I20260807 19:36:52.877803 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:52.880175 370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368: Bootstrap starting.
I20260807 19:36:52.883924 370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:52.885105 370 log.cc:824] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:52.885440 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:52.882510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50973 (local address 127.8.216.190:33507)
0807 19:36:52.882918 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:52.882926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:52.883254 (+ 328us) server_negotiation.cc:407] Connection header received
0807 19:36:52.883488 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:52.883493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:52.883545 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:52.883654 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:52.884711 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:52.884717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:52.884722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:52.884741 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:52.884798 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:52.884817 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:52.884948 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:52.885088 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 19:36:52.885161 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260807 19:36:52.888586 370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368: No bootstrap required, opened a new log
I20260807 19:36:52.896765 370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } }
I20260807 19:36:52.897208 370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:52.897336 370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f6c784a3c84c2f915fe2c2b0f66368, State: Initialized, Role: FOLLOWER
I20260807 19:36:52.898002 370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } }
I20260807 19:36:52.898257 370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:52.898413 370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:52.898607 370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:52.901140 370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } }
I20260807 19:36:52.901748 370 leader_election.cc:302] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01f6c784a3c84c2f915fe2c2b0f66368; no voters:
I20260807 19:36:52.902456 370 leader_election.cc:288] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:52.902550 381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:52.903415 381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [term 1 LEADER]: Becoming Leader. State: Replica: 01f6c784a3c84c2f915fe2c2b0f66368, State: Running, Role: LEADER
I20260807 19:36:52.904036 381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } }
I20260807 19:36:52.904953 370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:52.906844 382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } } }
I20260807 19:36:52.907589 382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:52.910598 383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01f6c784a3c84c2f915fe2c2b0f66368. Latest consensus state: current_term: 1 leader_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f6c784a3c84c2f915fe2c2b0f66368" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 33507 } } }
I20260807 19:36:52.911103 383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:52.911438 389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:52.916419 389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:52.926455 389 catalog_manager.cc:1347] Generated new cluster ID: d24ce69a03624770b25069f4d0942ace
I20260807 19:36:52.926616 389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:52.940274 389 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:52.940802 389 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:52.948596 389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01f6c784a3c84c2f915fe2c2b0f66368: Generated new TSK 0
I20260807 19:36:52.949791 389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:52.962282 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32765
W20260807 19:36:52.963560 365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38269
W20260807 19:36:52.964427 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 32765: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:52.964599 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 32765
2026-08-07T19:36:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2099 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:53 dist-test-slave-8w9v krb5kdc[403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:53 dist-test-slave-8w9v krb5kdc[403](info): set up 2 sockets
Aug 07 19:36:53 dist-test-slave-8w9v krb5kdc[403](info): commencing operation
krb5kdc: starting...
W20260807 19:36:54.150835 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:54 dist-test-slave-8w9v krb5kdc[403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:54.511830 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46389
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:54.875411 416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:54.875792 416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:54.875978 416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:54.883873 416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:54.884069 416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:54.884183 416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:54.884290 416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:54.884390 416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:54.895588 416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:54.897135 416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:54.898520 416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:54.911515 421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:54.911460 422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:54.911420 424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:54.912215 416 server_base.cc:1034] running on GCE node
I20260807 19:36:54.913182 416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:54.914753 416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:54.916059 416 hybrid_clock.cc:648] HybridClock initialized: now 1786131414915998 us; error 62 us; skew 500 ppm
I20260807 19:36:54.916596 416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:54 dist-test-slave-8w9v krb5kdc[403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:54.920358 416 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:54.922058 416 webserver.cc:460] Webserver started at http://127.8.216.190:38997/ using document root <none> and password file <none>
I20260807 19:36:54.922808 416 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:54.922955 416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:54.923290 416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:54.926600 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "dec2fd2aca094817a1b39eab3f84087f"
format_stamp: "Formatted at 2026-08-07 19:36:54 on dist-test-slave-8w9v"
server_key: "8014087d06f9b72b85546177350e2579"
server_key_iv: "60dde833ac5009778bf6c28f6420fc2a"
server_key_version: "encryptionkey@0"
I20260807 19:36:54.927415 416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "dec2fd2aca094817a1b39eab3f84087f"
format_stamp: "Formatted at 2026-08-07 19:36:54 on dist-test-slave-8w9v"
server_key: "8014087d06f9b72b85546177350e2579"
server_key_iv: "60dde833ac5009778bf6c28f6420fc2a"
server_key_version: "encryptionkey@0"
I20260807 19:36:54.933115 416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:36:54.937089 432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:54.938506 416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:36:54.938740 416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "dec2fd2aca094817a1b39eab3f84087f"
format_stamp: "Formatted at 2026-08-07 19:36:54 on dist-test-slave-8w9v"
server_key: "8014087d06f9b72b85546177350e2579"
server_key_iv: "60dde833ac5009778bf6c28f6420fc2a"
server_key_version: "encryptionkey@0"
I20260807 19:36:54.938941 416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:54.954747 416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:54.959475 416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:54.959790 416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:54.979475 416 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:46389
I20260807 19:36:54.979494 490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:46389 every 8 connection(s)
I20260807 19:36:54.980999 416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:54.984871 491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:54.987525 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 416
I20260807 19:36:54.987711 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:54.988298 9058 external_mini_cluster.cc:1428] Setting key aa3e22572cd39d01af7e4b5d1f240f53
I20260807 19:36:54.998301 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f: Bootstrap starting.
I20260807 19:36:55.002676 491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f: Neither blocks nor log segments found. Creating new log.
Aug 07 19:36:54 dist-test-slave-8w9v krb5kdc[403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:36:55.004071 491 log.cc:824] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:55.007477 491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f: No bootstrap required, opened a new log
I20260807 19:36:55.013027 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:54.990271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49279 (local address 127.8.216.190:46389)
0807 19:36:54.991136 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:54.991148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:54.991197 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:36:54.992993 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:54.993017 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:54.993438 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:54.993762 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:54.995701 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:54.997162 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:54.998959 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:54.999646 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:54.999778 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:55.005123 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:55.005133 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:55.005152 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:55.005193 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:55.008619 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:55.009400 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:55.009407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:55.009413 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:55.009593 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:55.010004 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:55.010009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:55.010014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:55.010603 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:55.010947 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:55.011395 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 19:36:55.011872 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":387,"threads_started":1}
I20260807 19:36:55.015834 491 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } }
I20260807 19:36:55.016408 491 raft_consensus.cc:374] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:55.016535 491 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec2fd2aca094817a1b39eab3f84087f, State: Initialized, Role: FOLLOWER
I20260807 19:36:55.017184 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:55.017199 491 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } }
I20260807 19:36:55.017513 491 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:55.017671 491 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:55.017863 491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:55.020304 491 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } }
I20260807 19:36:55.020900 491 leader_election.cc:302] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec2fd2aca094817a1b39eab3f84087f; no voters:
I20260807 19:36:55.021646 491 leader_election.cc:288] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:55.021843 501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:55.022637 501 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [term 1 LEADER]: Becoming Leader. State: Replica: dec2fd2aca094817a1b39eab3f84087f, State: Running, Role: LEADER
I20260807 19:36:55.023492 501 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } }
I20260807 19:36:55.026081 501 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec2fd2aca094817a1b39eab3f84087f. Latest consensus state: current_term: 1 leader_uuid: "dec2fd2aca094817a1b39eab3f84087f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } } }
I20260807 19:36:55.026448 501 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:55.027848 504 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dec2fd2aca094817a1b39eab3f84087f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec2fd2aca094817a1b39eab3f84087f" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 46389 } } }
I20260807 19:36:55.028301 504 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:55.030864 491 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:55.031127 506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:55.045394 506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:55.052649 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:55.021971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45549 (local address 127.8.216.190:46389)
0807 19:36:55.022182 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:55.022195 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:55.031780 (+ 9585us) server_negotiation.cc:407] Connection header received
0807 19:36:55.032016 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:55.032022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:55.032109 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:55.032236 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:55.035033 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:55.036663 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:55.039298 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:55.040332 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:55.040485 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:55.044840 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:55.044847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:55.044855 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:36:55.044915 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:55.048785 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:55.049811 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:55.049818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:55.049829 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:55.050016 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:36:55.050549 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:36:55.050555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:36:55.050558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:36:55.051038 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:55.051305 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:55.051970 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 19:36:55.052262 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:36:55.059665 506 catalog_manager.cc:1347] Generated new cluster ID: 42d51ad67a7b418d93bd8eccb4beaf4a
I20260807 19:36:55.059827 506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:55.083258 506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:55.085397 506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:55.096915 506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dec2fd2aca094817a1b39eab3f84087f: Generated new TSK 0
I20260807 19:36:55.097900 506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:55.113757 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 416
I20260807 19:36:55.115126 485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:36:55.210431 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 416
2026-08-07T19:36:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2246 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:55 dist-test-slave-8w9v krb5kdc[524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:55 dist-test-slave-8w9v krb5kdc[524](info): set up 2 sockets
Aug 07 19:36:55 dist-test-slave-8w9v krb5kdc[524](info): commencing operation
krb5kdc: starting...
W20260807 19:36:56.427568 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:56 dist-test-slave-8w9v krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:56.779933 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42953
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:42953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:57.143497 539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:57.143903 539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:57.144078 539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:57.151826 539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:57.152034 539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:57.152185 539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:57.152298 539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:57.152390 539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:57.163589 539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:42953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:42953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:57.165158 539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:57.166602 539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:57.179742 544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:57.179675 545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:57.179701 547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:57.180868 539 server_base.cc:1034] running on GCE node
I20260807 19:36:57.181843 539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:57.183485 539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:57.184733 539 hybrid_clock.cc:648] HybridClock initialized: now 1786131417184697 us; error 57 us; skew 500 ppm
I20260807 19:36:57.185129 539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:57 dist-test-slave-8w9v krb5kdc[524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131417, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:57.194310 539 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:57.196053 539 webserver.cc:460] Webserver started at http://127.8.216.190:40923/ using document root <none> and password file <none>
I20260807 19:36:57.196833 539 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:57.197005 539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:57.197340 539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:57.200609 539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "ea51f6da42034ac3892c16133c44af08"
format_stamp: "Formatted at 2026-08-07 19:36:57 on dist-test-slave-8w9v"
server_key: "25efe8fa1bb7a93185ee71a112899290"
server_key_iv: "5208314aa2172ae41b78da73f38787eb"
server_key_version: "encryptionkey@0"
I20260807 19:36:57.201462 539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "ea51f6da42034ac3892c16133c44af08"
format_stamp: "Formatted at 2026-08-07 19:36:57 on dist-test-slave-8w9v"
server_key: "25efe8fa1bb7a93185ee71a112899290"
server_key_iv: "5208314aa2172ae41b78da73f38787eb"
server_key_version: "encryptionkey@0"
I20260807 19:36:57.207060 539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:36:57.210970 554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:57.212500 539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:36:57.212711 539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "ea51f6da42034ac3892c16133c44af08"
format_stamp: "Formatted at 2026-08-07 19:36:57 on dist-test-slave-8w9v"
server_key: "25efe8fa1bb7a93185ee71a112899290"
server_key_iv: "5208314aa2172ae41b78da73f38787eb"
server_key_version: "encryptionkey@0"
I20260807 19:36:57.212952 539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:57.238248 539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:57.243203 539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:57.243542 539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:57.263614 539 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:42953
I20260807 19:36:57.263633 607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:42953 every 8 connection(s)
I20260807 19:36:57.265276 539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:57.265836 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 539
I20260807 19:36:57.266037 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:57.266786 9058 external_mini_cluster.cc:1428] Setting key 0fc5c2d0319d831bafc45b8b38a3b8ba
I20260807 19:36:57.270655 608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:57.283257 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:57.268844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33451 (local address 127.8.216.190:42953)
0807 19:36:57.269852 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:57.269869 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:57.269929 (+ 60us) server_negotiation.cc:407] Connection header received
0807 19:36:57.272387 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:57.272418 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:57.272932 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:57.273417 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:57.273999 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:57.276140 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:57.277499 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:57.278593 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:57.278778 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:57.280091 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:57.280102 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:57.280152 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:57.280973 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:57.281074 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:57.281115 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:57.281239 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:57.281468 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 19:36:57.281786 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":437,"threads_started":1}
I20260807 19:36:57.285261 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08: Bootstrap starting.
I20260807 19:36:57.287477 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:57.289044 608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:57.290494 608 log.cc:824] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:57.294013 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08: No bootstrap required, opened a new log
I20260807 19:36:57.300174 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:57.293271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34533 (local address 127.8.216.190:42953)
0807 19:36:57.293482 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:57.293489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:57.293941 (+ 452us) server_negotiation.cc:407] Connection header received
0807 19:36:57.294109 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:57.294114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:57.294193 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:57.294308 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:57.294862 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:57.295947 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:57.297340 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:36:57.298088 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:36:57.298228 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:36:57.299313 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:57.299320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:57.299325 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:57.299349 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:57.299406 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:57.299428 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:57.299530 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:57.299659 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:36:57.299767 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:36:57.303565 608 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } }
I20260807 19:36:57.303973 608 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:57.304107 608 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea51f6da42034ac3892c16133c44af08, State: Initialized, Role: FOLLOWER
I20260807 19:36:57.304808 608 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } }
I20260807 19:36:57.305058 608 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:57.305204 608 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:57.305395 608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:57.307700 608 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } }
I20260807 19:36:57.308337 608 leader_election.cc:302] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea51f6da42034ac3892c16133c44af08; no voters:
I20260807 19:36:57.308974 608 leader_election.cc:288] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:57.309116 618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:57.310225 618 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [term 1 LEADER]: Becoming Leader. State: Replica: ea51f6da42034ac3892c16133c44af08, State: Running, Role: LEADER
I20260807 19:36:57.310745 618 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } }
I20260807 19:36:57.311079 608 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:57.313769 620 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea51f6da42034ac3892c16133c44af08. Latest consensus state: current_term: 1 leader_uuid: "ea51f6da42034ac3892c16133c44af08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } } }
I20260807 19:36:57.314260 620 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:57.314913 619 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea51f6da42034ac3892c16133c44af08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea51f6da42034ac3892c16133c44af08" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 42953 } } }
I20260807 19:36:57.315258 619 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:57.316432 627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:57.321722 627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:57.331497 627 catalog_manager.cc:1347] Generated new cluster ID: 5ece82c8d6f8446788c5b6d404fc2080
I20260807 19:36:57.331638 627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:57.355806 627 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:57.357268 627 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:57.378984 627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea51f6da42034ac3892c16133c44af08: Generated new TSK 0
I20260807 19:36:57.380041 627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:57.396939 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 539
W20260807 19:36:57.398612 602 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33451
W20260807 19:36:57.399426 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 539: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:57.399600 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 539
2026-08-07T19:36:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:57 dist-test-slave-8w9v krb5kdc[644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:57 dist-test-slave-8w9v krb5kdc[644](info): set up 2 sockets
Aug 07 19:36:57 dist-test-slave-8w9v krb5kdc[644](info): commencing operation
krb5kdc: starting...
W20260807 19:36:58.599579 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:36:58 dist-test-slave-8w9v krb5kdc[644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:36:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:36:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:36:58.960340 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38995
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:36849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:36:59.325472 659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:36:59.325804 659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:36:59.326001 659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:36:59.333570 659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:36:59.333711 659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:36:59.333760 659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:36:59.333871 659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:36:59.333958 659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:36:59.345014 659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:36849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:36:59.346446 659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:36:59.347823 659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:36:59.360575 664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:59.360597 665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:36:59.360630 667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:36:59.361950 659 server_base.cc:1034] running on GCE node
I20260807 19:36:59.362790 659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:36:59.364329 659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:36:59.365581 659 hybrid_clock.cc:648] HybridClock initialized: now 1786131419365512 us; error 67 us; skew 500 ppm
I20260807 19:36:59.365964 659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:36:59 dist-test-slave-8w9v krb5kdc[644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:36:59.369396 659 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:36:59.371105 659 webserver.cc:460] Webserver started at http://127.8.216.190:35857/ using document root <none> and password file <none>
I20260807 19:36:59.371850 659 fs_manager.cc:359] Metadata directory not provided
I20260807 19:36:59.372004 659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:36:59.372383 659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:36:59.375727 659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "409e104bf896494ca3480e12a9abb329"
format_stamp: "Formatted at 2026-08-07 19:36:59 on dist-test-slave-8w9v"
server_key: "beceead79f61f10a4edea41f537fa7b5"
server_key_iv: "6dbdba93df552daa592f4c7c07014e9d"
server_key_version: "encryptionkey@0"
I20260807 19:36:59.376592 659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "409e104bf896494ca3480e12a9abb329"
format_stamp: "Formatted at 2026-08-07 19:36:59 on dist-test-slave-8w9v"
server_key: "beceead79f61f10a4edea41f537fa7b5"
server_key_iv: "6dbdba93df552daa592f4c7c07014e9d"
server_key_version: "encryptionkey@0"
I20260807 19:36:59.382050 659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:36:59.385844 674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:36:59.387265 659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 19:36:59.387478 659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "409e104bf896494ca3480e12a9abb329"
format_stamp: "Formatted at 2026-08-07 19:36:59 on dist-test-slave-8w9v"
server_key: "beceead79f61f10a4edea41f537fa7b5"
server_key_iv: "6dbdba93df552daa592f4c7c07014e9d"
server_key_version: "encryptionkey@0"
I20260807 19:36:59.387713 659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:36:59.394109 659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:36:59.398725 659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:36:59.399075 659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:36:59.418843 659 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:38995
I20260807 19:36:59.418877 728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:38995 every 8 connection(s)
I20260807 19:36:59.420352 659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:36:59.424098 729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:36:59.426055 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 659
I20260807 19:36:59.426246 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:36:59.426877 9058 external_mini_cluster.cc:1428] Setting key 94e4c0fdb54bdb2064f48e3579558d9f
I20260807 19:36:59.434756 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:59.428799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58301 (local address 127.8.216.190:38995)
0807 19:36:59.429660 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:59.429676 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:59.429723 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:36:59.431316 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:59.431342 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:59.431714 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:59.432053 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:59.433184 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:59.433190 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:59.433203 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:59.433218 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:59.433277 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:59.433296 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:59.433497 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:59.433594 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 19:36:59.433797 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":433,"threads_started":1}
I20260807 19:36:59.438426 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329: Bootstrap starting.
I20260807 19:36:59.438700 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:36:59.441767 729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329: Neither blocks nor log segments found. Creating new log.
I20260807 19:36:59.442996 729 log.cc:824] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329: Log is configured to *not* fsync() on all Append() calls
I20260807 19:36:59.445827 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:36:59.443096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56095 (local address 127.8.216.190:38995)
0807 19:36:59.443309 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:36:59.443318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:36:59.443753 (+ 435us) server_negotiation.cc:407] Connection header received
0807 19:36:59.443958 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:36:59.443962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:36:59.444012 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:36:59.444093 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:36:59.445164 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:36:59.445169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:36:59.445173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:36:59.445191 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:36:59.445240 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:36:59.445263 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:36:59.445392 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:36:59.445465 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 19:36:59.445528 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:36:59.446542 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329: No bootstrap required, opened a new log
I20260807 19:36:59.454218 729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } }
I20260807 19:36:59.454649 729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:36:59.454761 729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409e104bf896494ca3480e12a9abb329, State: Initialized, Role: FOLLOWER
I20260807 19:36:59.455507 729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } }
I20260807 19:36:59.455812 729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:36:59.455997 729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:36:59.456218 729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:36:59.458535 729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } }
I20260807 19:36:59.459074 729 leader_election.cc:302] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409e104bf896494ca3480e12a9abb329; no voters:
I20260807 19:36:59.459682 729 leader_election.cc:288] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:36:59.459849 740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:36:59.460598 740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [term 1 LEADER]: Becoming Leader. State: Replica: 409e104bf896494ca3480e12a9abb329, State: Running, Role: LEADER
I20260807 19:36:59.461199 740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } }
I20260807 19:36:59.462034 729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:36:59.465231 742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409e104bf896494ca3480e12a9abb329. Latest consensus state: current_term: 1 leader_uuid: "409e104bf896494ca3480e12a9abb329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } } }
I20260807 19:36:59.466029 742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:59.465260 741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "409e104bf896494ca3480e12a9abb329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e104bf896494ca3480e12a9abb329" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 38995 } } }
I20260807 19:36:59.466835 741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329 [sys.catalog]: This master's current role is: LEADER
I20260807 19:36:59.469009 751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:36:59.474401 751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:36:59.485146 751 catalog_manager.cc:1347] Generated new cluster ID: f13fa532bc1a4058bb2de56be9306da2
I20260807 19:36:59.485280 751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:36:59.514633 751 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:36:59.515172 751 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:36:59.526055 751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 409e104bf896494ca3480e12a9abb329: Generated new TSK 0
I20260807 19:36:59.527215 751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:36:59.536911 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 659
W20260807 19:36:59.538307 722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58301
W20260807 19:36:59.539197 9058 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 659: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:36:59.539412 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 659
2026-08-07T19:36:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2134 ms)
[----------] 18 tests from AuthTokenIssuingTest (39146 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:36:59 dist-test-slave-8w9v krb5kdc[763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:36:59 dist-test-slave-8w9v krb5kdc[763](info): set up 2 sockets
Aug 07 19:36:59 dist-test-slave-8w9v krb5kdc[763](info): commencing operation
krb5kdc: starting...
W20260807 19:37:00.734668 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:37:00 dist-test-slave-8w9v krb5kdc[763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:37:01.096087 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44877
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:01.467299 776 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:01.467754 776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:01.467885 776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:01.468039 776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:01.476018 776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:01.476146 776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:01.476241 776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:01.476320 776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:01.476445 776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:01.486888 776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:44877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:01.488447 776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:01.489945 776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:01.503130 787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:01.503168 784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:01.503106 783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:01.504674 776 server_base.cc:1034] running on GCE node
I20260807 19:37:01.505615 776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:01.507169 776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:01.508430 776 hybrid_clock.cc:648] HybridClock initialized: now 1786131421508359 us; error 43 us; skew 500 ppm
I20260807 19:37:01.508791 776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:01 dist-test-slave-8w9v krb5kdc[763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:01.512271 776 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:37:01.513955 776 webserver.cc:460] Webserver started at http://127.8.216.190:40283/ using document root <none> and password file <none>
I20260807 19:37:01.514731 776 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:01.514876 776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:01.515244 776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:01.518623 776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "4acc680a84a541a5b64eb2f4cf246fca"
format_stamp: "Formatted at 2026-08-07 19:37:01 on dist-test-slave-8w9v"
server_key: "267131f54f6aa00dc46480b90f0fc3d1"
server_key_iv: "ae587cb4cfd1d7c35199911669228619"
server_key_version: "encryptionkey@0"
I20260807 19:37:01.519492 776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "4acc680a84a541a5b64eb2f4cf246fca"
format_stamp: "Formatted at 2026-08-07 19:37:01 on dist-test-slave-8w9v"
server_key: "267131f54f6aa00dc46480b90f0fc3d1"
server_key_iv: "ae587cb4cfd1d7c35199911669228619"
server_key_version: "encryptionkey@0"
I20260807 19:37:01.525220 776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:37:01.529031 795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:01.530428 776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:01.530630 776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "4acc680a84a541a5b64eb2f4cf246fca"
format_stamp: "Formatted at 2026-08-07 19:37:01 on dist-test-slave-8w9v"
server_key: "267131f54f6aa00dc46480b90f0fc3d1"
server_key_iv: "ae587cb4cfd1d7c35199911669228619"
server_key_version: "encryptionkey@0"
I20260807 19:37:01.530869 776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:01.550953 776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:01.555995 776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:01.556352 776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:01.577252 776 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:44877
I20260807 19:37:01.577275 849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:44877 every 8 connection(s)
I20260807 19:37:01.578797 776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:01.583035 850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:01.583344 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 776
I20260807 19:37:01.583575 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:37:01.584512 9058 external_mini_cluster.cc:1428] Setting key 0c5b1bdf65408a27ee4eaa932525e9fb
I20260807 19:37:01.594151 850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:01 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.189@KRBTEST.COM; defaulting to no policy
I20260807 19:37:01.621834 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.598687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.8.216.190:44877)
0807 19:37:01.600643 (+ 1956us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:01.600651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:01.600704 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:37:01.602802 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:01.602809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:01.603059 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:01.603240 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:01.605097 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:01.606329 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:01.608735 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:01.609318 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:01.609485 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:01.613926 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:01.613936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:01.613950 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:01.613987 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:01.617076 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:01.617872 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:01.617879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:01.617881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:01.618049 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:01.618481 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:01.618490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:01.618492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:01.619052 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:01.619272 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:01.619828 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 19:37:01.620239 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1702,"spinlock_wait_cycles":18176,"thread_start_us":1456,"threads_started":1}
I20260807 19:37:01.621853 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.598687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44877 (local address 127.0.0.1:39684)
0807 19:37:01.599708 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.599761 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:01.602372 (+ 2611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:01.603333 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:01.603342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:01.604249 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:01.604847 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:01.604889 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:01.606510 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:01.606519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:01.608580 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:01.608600 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:01.609587 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:01.609597 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:01.609836 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:01.610647 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:01.610683 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:01.613637 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:01.617247 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:01.617260 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:01.617275 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:01.617713 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:01.618164 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:01.618171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:01.618174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:01.618341 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:01.619206 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:01.619214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:01.619435 (+ 221us) client_negotiation.cc:769] Sending connection context
0807 19:37:01.619689 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 19:37:01.620383 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":199,"thread_start_us":449,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.189@KRBTEST.COM" created.
I20260807 19:37:01.625206 850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:37:01.626689 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.626127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43624)
0807 19:37:01.626350 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.626514 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 19:37:01.627045 796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
I20260807 19:37:01.628896 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.628387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43628)
0807 19:37:01.628581 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.628684 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 19:37:01.629132 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:46869: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
I20260807 19:37:01.665169 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:46869
--webserver_interface=127.8.216.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:37:01.678524 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } attempt: 1
I20260807 19:37:01.679796 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.679323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43634)
0807 19:37:01.679534 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.679661 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:37:01.680963 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.680586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43640)
0807 19:37:01.680739 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.680819 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 19:37:01.681176 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:46869: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
I20260807 19:37:01.749478 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } attempt: 2
I20260807 19:37:01.751318 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.750550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43652)
0807 19:37:01.750953 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.751122 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:37:01.753039 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.752316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43666)
0807 19:37:01.752604 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.752735 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 19:37:01.753410 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:46869: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
I20260807 19:37:01.844740 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } attempt: 3
I20260807 19:37:01.846324 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.845618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43672)
0807 19:37:01.845923 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.846081 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 19:37:01.847725 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.847179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43678)
0807 19:37:01.847379 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.847502 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:37:01.847992 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:46869: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
I20260807 19:37:01.991293 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } attempt: 4
I20260807 19:37:01.992779 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.992058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43682)
0807 19:37:01.992449 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.992575 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260807 19:37:01.994117 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:01.993641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43684)
0807 19:37:01.993836 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:01.993945 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:37:01.994414 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:46869: Network error: Client connection negotiation failed: client connection to 127.8.216.189:46869: connect: Connection refused (error 111)
W20260807 19:37:02.044971 857 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:02.045393 857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:02.045516 857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:02.045655 857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:02.053354 857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:02.053458 857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:02.053519 857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:02.053592 857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:02.053691 857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:02.064069 857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.216.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:02.065500 857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:02.066862 857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:02.079428 862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:02.079396 863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:02.079371 865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:02.080161 857 server_base.cc:1034] running on GCE node
I20260807 19:37:02.081022 857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:02.082592 857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:02.083896 857 hybrid_clock.cc:648] HybridClock initialized: now 1786131422083818 us; error 39 us; skew 500 ppm
I20260807 19:37:02.084396 857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:02.087848 857 init.cc:377] Logged in from keytab as kudu/127.8.216.189@KRBTEST.COM (short username kudu)
I20260807 19:37:02.089573 857 webserver.cc:460] Webserver started at http://127.8.216.189:45053/ using document root <none> and password file <none>
I20260807 19:37:02.090327 857 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:02.090466 857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:02.090803 857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:02.094017 857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data/instance:
uuid: "ebe18aa97d594856955adde3d52cc034"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "61a2328f0f55bc1749086178e723aa7b"
server_key_iv: "c4c8c7db3a30a1d7e18bcf712cdc95fb"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.094874 857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal/instance:
uuid: "ebe18aa97d594856955adde3d52cc034"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "61a2328f0f55bc1749086178e723aa7b"
server_key_iv: "c4c8c7db3a30a1d7e18bcf712cdc95fb"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.100586 857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:37:02.104416 872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:02.105908 857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:02.106154 857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "ebe18aa97d594856955adde3d52cc034"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "61a2328f0f55bc1749086178e723aa7b"
server_key_iv: "c4c8c7db3a30a1d7e18bcf712cdc95fb"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.106422 857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:02.129392 857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:02.133836 857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:02.134145 857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:02.154633 857 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.189:46869
I20260807 19:37:02.154652 926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.189:46869 every 8 connection(s)
I20260807 19:37:02.156212 857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
I20260807 19:37:02.160329 927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:02.162113 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 857
I20260807 19:37:02.162334 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-1/wal/instance
I20260807 19:37:02.162912 9058 external_mini_cluster.cc:1428] Setting key 4b8818a5257f963d63224b52cd098051
I20260807 19:37:02.170849 927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.216.188@KRBTEST.COM; defaulting to no policy
I20260807 19:37:02.200515 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.174738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39688 (local address 127.8.216.190:44877)
0807 19:37:02.175341 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.175348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.176564 (+ 1216us) server_negotiation.cc:407] Connection header received
0807 19:37:02.179371 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.179380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.179492 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.179638 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.181396 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.182638 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.184446 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.185227 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.185400 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.190122 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.190128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.190131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.190183 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.196540 (+ 6357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.197827 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.197832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.197834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.198000 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.198500 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.198504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.198507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.198846 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.198978 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.199844 (+ 866us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.200046 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":288,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.188@KRBTEST.COM" created.
I20260807 19:37:02.201808 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.175385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44877 (local address 127.0.0.1:39688)
0807 19:37:02.176334 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.176413 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.178988 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.179854 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.179863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.180533 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.181150 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.181169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.182852 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.182856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.184290 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.184304 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.185515 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.185525 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.185890 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.186726 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.186751 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.189795 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.196766 (+ 6971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.196779 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.196801 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.197672 (+ 871us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.198155 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.198158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.198161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.198359 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.198992 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.199008 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.199409 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.199685 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.200180 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":331,"threads_started":1}
I20260807 19:37:02.204002 927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:02.221873 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.205182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43690 (local address 127.8.216.189:46869)
0807 19:37:02.206018 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.206029 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.206065 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:37:02.206125 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.206135 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.206409 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.206574 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.207306 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.209076 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.210289 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.211282 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.211436 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.214898 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.214916 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.214939 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.214988 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.218569 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.219510 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.219517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.219519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.219638 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.220166 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.220174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.220180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.220581 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.220759 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.221338 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.221559 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":617,"thread_start_us":446,"threads_started":1}
I20260807 19:37:02.222187 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.204983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43690)
0807 19:37:02.205205 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.205228 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.205353 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.206581 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.206584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.206976 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.207125 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.207134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.209214 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.209218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.210135 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.210147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.211435 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.211439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.211632 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.212306 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.212324 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.214660 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.218760 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.218768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.218776 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.219361 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.219764 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.219772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.219778 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.219981 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.220723 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.220739 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.220915 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.221080 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.221441 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":36}
I20260807 19:37:02.224905 927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:37:02.226316 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.225669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59000)
0807 19:37:02.225886 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.226069 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 19:37:02.226600 874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.228439 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.227932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59006)
0807 19:37:02.228150 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.228260 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 19:37:02.228765 927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
I20260807 19:37:02.247220 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:42835
--webserver_interface=127.8.216.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:35157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:37:02.278178 927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 1
I20260807 19:37:02.279588 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.279041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59018)
0807 19:37:02.279282 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.279405 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 19:37:02.280956 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.280531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59022)
0807 19:37:02.280718 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.280832 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:37:02.281291 927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.293735 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } attempt: 5
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:02.309636 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.294714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.8.216.189:46869)
0807 19:37:02.294909 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.294913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.294956 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:37:02.295226 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.295231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.295310 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.295406 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.296501 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.297232 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.298920 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.299403 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.299521 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.303465 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.303471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.303474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.303513 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.306463 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.307346 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.307353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.307357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.307523 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.307975 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.307981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.307987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.308385 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.308558 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.309008 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.309262 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:37:02.309657 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.294566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43702)
0807 19:37:02.294856 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.294881 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.295014 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.295477 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.295481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.296045 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.296320 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.296334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.297382 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.297387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.298738 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.298765 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.299569 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.299575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.299818 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.300553 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.300580 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.303173 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.306716 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.306723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.306728 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.307194 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.307670 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.307676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.307680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.307838 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.308529 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.308534 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.308712 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.308871 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.309248 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 19:37:02.310927 850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:37:02.312240 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.311730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59036)
0807 19:37:02.311928 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.312017 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:37:02.313306 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.312932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59042)
0807 19:37:02.313095 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.313197 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:37:02.313542 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.349568 927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 2
I20260807 19:37:02.350834 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.350313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59048)
0807 19:37:02.350552 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.350656 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 19:37:02.351898 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.351532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59056)
0807 19:37:02.351689 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.351769 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:37:02.352195 927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.364799 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 1
I20260807 19:37:02.366101 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.365611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59062)
0807 19:37:02.365823 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.365924 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:37:02.367282 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.366913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59066)
0807 19:37:02.367097 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.367170 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 19:37:02.367524 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.435752 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 2
I20260807 19:37:02.437062 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.436516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59074)
0807 19:37:02.436779 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.436871 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:37:02.438215 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.437828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59082)
0807 19:37:02.438024 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.438094 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 19:37:02.438443 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.443401 927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 3
I20260807 19:37:02.444504 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.444023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59098)
0807 19:37:02.444242 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.444336 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:37:02.445595 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.445152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59112)
0807 19:37:02.445394 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.445473 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260807 19:37:02.445822 927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.544667 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 3
I20260807 19:37:02.545933 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.545391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59124)
0807 19:37:02.545641 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.545744 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 19:37:02.547106 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.546690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59138)
0807 19:37:02.546925 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.546999 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 19:37:02.547328 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.589038 927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 4
I20260807 19:37:02.590438 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.589790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59140)
0807 19:37:02.590096 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.590217 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:37:02.591832 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.591374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59150)
0807 19:37:02.591600 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.591683 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260807 19:37:02.592111 927 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
W20260807 19:37:02.628949 935 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:02.629333 935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:02.629462 935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:02.629588 935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:02.637473 935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:02.637567 935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:02.637665 935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:02.637759 935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:02.637872 935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:02.648317 935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:35157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:44877,127.8.216.189:46869,127.8.216.188:42835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:42835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.216.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:02.649688 935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:02.651119 935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:02.663394 941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:02.663444 943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:02.663479 940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:02.664424 935 server_base.cc:1034] running on GCE node
I20260807 19:37:02.665305 935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:02.666788 935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:02.668064 935 hybrid_clock.cc:648] HybridClock initialized: now 1786131422667986 us; error 44 us; skew 500 ppm
I20260807 19:37:02.668468 935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:02.671747 935 init.cc:377] Logged in from keytab as kudu/127.8.216.188@KRBTEST.COM (short username kudu)
I20260807 19:37:02.673537 935 webserver.cc:460] Webserver started at http://127.8.216.188:44225/ using document root <none> and password file <none>
I20260807 19:37:02.674273 935 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:02.674430 935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:02.674870 935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:02.678006 935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data/instance:
uuid: "860495e3906945b686116e48bb5c89ac"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "662078ba7121c5bcec2f9ab488dd4e98"
server_key_iv: "9c7036eb210be4952406595994a57083"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.678772 935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal/instance:
uuid: "860495e3906945b686116e48bb5c89ac"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "662078ba7121c5bcec2f9ab488dd4e98"
server_key_iv: "9c7036eb210be4952406595994a57083"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.684576 935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:37:02.688104 951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:02.689564 935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:37:02.689755 935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "860495e3906945b686116e48bb5c89ac"
format_stamp: "Formatted at 2026-08-07 19:37:02 on dist-test-slave-8w9v"
server_key: "662078ba7121c5bcec2f9ab488dd4e98"
server_key_iv: "9c7036eb210be4952406595994a57083"
server_key_version: "encryptionkey@0"
I20260807 19:37:02.689997 935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:02.705931 935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:02.710363 935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:02.710707 935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:02.724577 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 4
I20260807 19:37:02.725765 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.725320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59154)
0807 19:37:02.725522 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.725627 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:37:02.727041 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.726574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59164)
0807 19:37:02.726746 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.726851 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:37:02.727308 850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:42835: Network error: Client connection negotiation failed: client connection to 127.8.216.188:42835: connect: Connection refused (error 111)
I20260807 19:37:02.730811 935 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.188:42835
I20260807 19:37:02.730856 1005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.188:42835 every 8 connection(s)
I20260807 19:37:02.732297 935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
I20260807 19:37:02.735167 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 935
I20260807 19:37:02.735381 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131039239543-9058-0/minicluster-data/master-2/wal/instance
I20260807 19:37:02.735914 9058 external_mini_cluster.cc:1428] Setting key 4c0a52905b0bef96c605b09ea2f764b2
I20260807 19:37:02.736720 1006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:02.748091 1006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:37:02.761675 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.737928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.8.216.188:42835)
0807 19:37:02.738790 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.738808 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.738879 (+ 71us) server_negotiation.cc:407] Connection header received
0807 19:37:02.741065 (+ 2186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.741095 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.741669 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.742158 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.743510 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.745740 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.747085 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.748209 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.748443 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.752967 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.752978 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.752989 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.753056 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.756956 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.757812 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.757818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.757824 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.757985 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.758546 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.758551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.758554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.759299 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.759630 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.760015 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.760531 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":377,"threads_started":1}
I20260807 19:37:02.768232 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.751032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.8.216.190:44877)
0807 19:37:02.751557 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.751563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.752208 (+ 645us) server_negotiation.cc:407] Connection header received
0807 19:37:02.752468 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.752477 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.752584 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.752710 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.754462 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.755357 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.757415 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.758010 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.758162 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.761835 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.761842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.761847 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.761897 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.765204 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.765941 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.765946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.765950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.766096 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.766511 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.766516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.766519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.766808 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.767018 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.767521 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.767774 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":263,"threads_started":1}
I20260807 19:37:02.768225 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.751011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:44877 (local address 127.0.0.1:39694)
0807 19:37:02.752046 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.752091 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.752307 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.752838 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.752848 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.753853 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.754278 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.754301 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.755540 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.755546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.757266 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.757277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.758215 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.758220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.758339 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.759057 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.759078 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.761617 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.765347 (+ 3730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.765361 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.765368 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.765797 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.766216 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.766222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.766225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.766380 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.766963 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.766974 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.767174 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.767384 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.767744 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"thread_start_us":572,"threads_started":1}
I20260807 19:37:02.769941 1006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:02.781867 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.764744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.8.216.190:44877)
0807 19:37:02.765425 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.765430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.765451 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:02.765491 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.765494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.765547 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.765668 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.766970 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.767831 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.769600 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.770071 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.770265 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.775474 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.775482 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.775486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.775528 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.778734 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.779447 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.779456 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.779461 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.779636 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.780049 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.780056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.780061 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.780509 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.780818 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.781176 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.781430 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":339,"threads_started":1}
I20260807 19:37:02.785457 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.770833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:46869 (local address 127.0.0.1:43706)
0807 19:37:02.771051 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.771074 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.771171 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.771487 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.771491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.771971 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.772227 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.772241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.773227 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.773233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.774450 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.774463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.775474 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.775481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.775704 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.776361 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.776401 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.778770 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.782456 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.782465 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.782468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.782992 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.783422 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.783428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.783432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.783587 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.784408 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.784414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.784590 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.784746 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.785058 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:37:02.785480 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.770998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.8.216.189:46869)
0807 19:37:02.771182 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.771187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.771224 (+ 37us) server_negotiation.cc:407] Connection header received
0807 19:37:02.771273 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.771276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.771335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.771414 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.772379 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.773082 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.774608 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.775290 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.775436 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.778964 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.778969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.778972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.779006 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.782307 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.783146 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.783150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.783152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.783294 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.783749 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.783754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.783757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.784256 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.784447 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.784857 (+ 410us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.785103 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:37:02.786561 1006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:37:02.801128 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.787383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.8.216.188:42835)
0807 19:37:02.787588 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.787593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.787839 (+ 246us) server_negotiation.cc:407] Connection header received
0807 19:37:02.787942 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.787947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.788019 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.788146 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.788841 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.789650 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.790985 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.791743 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.791879 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.795669 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.795673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.795676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.795704 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.798491 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.799138 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.799146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.799149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.799270 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.799618 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.799625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.799629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.799901 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.800056 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.800435 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.800709 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":84}
I20260807 19:37:02.801270 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.783785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.8.216.189:46869)
0807 19:37:02.784445 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.784449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.784469 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:37:02.784508 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.784512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.784562 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.784680 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.786156 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.787255 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.788877 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.789290 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.789389 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.794741 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.794750 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.794757 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.794803 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.798283 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.799069 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.799075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.799079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.799228 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.799635 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.799640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.799643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.799949 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.800269 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.800576 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.800819 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":550,"thread_start_us":344,"threads_started":1}
I20260807 19:37:02.801414 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.787490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59180)
0807 19:37:02.787744 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.787769 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.787855 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.788212 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.788215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.788567 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.788701 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.788709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.789785 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.789788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.790839 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.790851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.791934 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.791939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.792092 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.792740 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.792767 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.795479 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.798640 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.798647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.798651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.799023 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.799380 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.799387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.799392 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.799518 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.800016 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.800020 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.800191 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.800348 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.800854 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":163}
I20260807 19:37:02.806442 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:02.807178 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: Bootstrap starting.
I20260807 19:37:02.810784 1006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:02.811911 1006 log.cc:824] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:02.809283 824 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4acc680a84a541a5b64eb2f4cf246fca"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39702
I20260807 19:37:02.814730 1006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: No bootstrap required, opened a new log
I20260807 19:37:02.822064 1006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:02.822472 1006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860495e3906945b686116e48bb5c89ac, State: Initialized, Role: FOLLOWER
I20260807 19:37:02.822950 1006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:02.823920 1015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:02.824366 1015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:02.825328 1006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:37:02.836201 1027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:37:02.836359 1027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:37:02.833473 935 master_runner.cc:186] Error getting master registration for 127.8.216.190:44877: OK, instance_id { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" instance_seqno: 1786131421571130 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:37:02.845504 935 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:37:02.891448 927 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 5
Aug 07 19:37:02 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:37:02.910149 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.892455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59186)
0807 19:37:02.892699 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:02.892725 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:02.892878 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:02.893528 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:02.893533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:02.894149 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:02.894385 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.894399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.895623 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.895629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.897181 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:02.897200 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.898390 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.898397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:02.898629 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.899422 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:02.899450 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:02.902561 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:02.907091 (+ 4530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.907096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.907099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.907580 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.908054 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:02.908059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:02.908063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:02.908274 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:02.909064 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:02.909068 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:02.909246 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:37:02.909395 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:37:02.909736 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:37:02.910229 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:02.892544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.8.216.188:42835)
0807 19:37:02.893154 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:02.893160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:02.893180 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:37:02.893245 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:02.893253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:02.893339 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:02.893450 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:02.894562 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.895452 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.897377 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:02.898187 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:02.898328 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:02.902831 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:02.902838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:02.902843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:02.902891 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:02.906871 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.907724 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.907731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.907735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.907923 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:02.908401 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:02.908407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:02.908411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:02.908935 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:02.909113 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:02.909553 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 19:37:02.909825 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445}
I20260807 19:37:02.916826 927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: Bootstrap starting.
I20260807 19:37:02.921341 927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:02.922791 927 log.cc:824] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:02.926968 927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: No bootstrap required, opened a new log
I20260807 19:37:02.939023 927 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:02.939520 927 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe18aa97d594856955adde3d52cc034, State: Initialized, Role: FOLLOWER
I20260807 19:37:02.940102 927 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:02.941287 1030 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:02.941784 1030 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:02.942806 927 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:37:02.953821 1042 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:37:02.953977 1042 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:37:02.950546 857 master_runner.cc:186] Error getting master registration for 127.8.216.190:44877: OK, instance_id { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" instance_seqno: 1786131421571130 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:37:02.962575 857 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:37:02.963155 959 tablet.cc:2369] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:03.004578 850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } attempt: 5
Aug 07 19:37:03 dist-test-slave-8w9v krb5kdc[763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:37:03.024349 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:03.005439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:42835 (local address 127.0.0.1:59194)
0807 19:37:03.005755 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:03.005776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:03.005917 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:03.006366 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:03.006372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:03.006932 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:03.007170 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:03.007184 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.008818 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.008825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:03.010425 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:03.010459 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.011792 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.011799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:03.012033 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:03.012921 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:03.012947 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:03.016183 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:03.021106 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:03.021114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:03.021118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:03.021728 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:03.022249 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:03.022255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:03.022259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:03.022470 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:03.023167 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:03.023173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:03.023370 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 19:37:03.023549 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 19:37:03.023878 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260807 19:37:03.024410 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:03.005550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.8.216.188:42835)
0807 19:37:03.005759 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.005766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.005846 (+ 80us) server_negotiation.cc:407] Connection header received
0807 19:37:03.006108 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.006114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.006199 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.006298 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.007395 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.008613 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.010657 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.011392 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.011538 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:03.016512 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:03.016519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:03.016524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:03.016574 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:03.020902 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.021925 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.021933 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.021937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.022122 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.022604 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.022610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.022613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.023045 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:03.023219 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:03.023673 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 19:37:03.023965 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:37:03.031441 850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca: Bootstrap starting.
I20260807 19:37:03.035873 850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:03.037422 850 log.cc:824] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:03.041775 850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca: No bootstrap required, opened a new log
I20260807 19:37:03.050055 850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:03.050525 850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acc680a84a541a5b64eb2f4cf246fca, State: Initialized, Role: FOLLOWER
I20260807 19:37:03.051074 850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:03.058166 1045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.058634 1045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:03.060724 850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:03.060817 824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:37:03.061188 824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:03.064941 824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:03.067699 902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acc680a84a541a5b64eb2f4cf246fca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ebe18aa97d594856955adde3d52cc034"
I20260807 19:37:03.068305 902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:03.071995 902 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acc680a84a541a5b64eb2f4cf246fca in term 1.
I20260807 19:37:03.072989 796 leader_election.cc:302] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acc680a84a541a5b64eb2f4cf246fca, ebe18aa97d594856955adde3d52cc034; no voters:
I20260807 19:37:03.073514 1045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:03.074766 980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acc680a84a541a5b64eb2f4cf246fca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "860495e3906945b686116e48bb5c89ac"
I20260807 19:37:03.075340 980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:03.079075 980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acc680a84a541a5b64eb2f4cf246fca in term 1.
I20260807 19:37:03.080518 824 leader_election.cc:288] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [CANDIDATE]: Term 1 election: Requested vote from peers ebe18aa97d594856955adde3d52cc034 (127.8.216.189:46869), 860495e3906945b686116e48bb5c89ac (127.8.216.188:42835)
I20260807 19:37:03.089299 1045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [term 1 LEADER]: Becoming Leader. State: Replica: 4acc680a84a541a5b64eb2f4cf246fca, State: Running, Role: LEADER
I20260807 19:37:03.090190 1045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } }
I20260807 19:37:03.129307 1047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acc680a84a541a5b64eb2f4cf246fca. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.129755 1047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: This master's current role is: LEADER
W20260807 19:37:03.137400 878 tablet.cc:2369] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:03.140511 1059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:03.159353 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:03.098702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.8.216.189:46869)
0807 19:37:03.098960 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.098970 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.120896 (+ 21926us) server_negotiation.cc:407] Connection header received
0807 19:37:03.121277 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.121286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.121401 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.121493 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.128286 (+ 6793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.129432 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.131611 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.132331 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.132439 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:03.141375 (+ 8936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:03.141383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:03.141388 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:03.141436 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:03.145900 (+ 4464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.156559 (+ 10659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.156566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.156570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.156755 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.157299 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.157304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.157308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.157740 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:03.158046 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:03.158629 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 19:37:03.158929 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:37:03.163913 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:03.114826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39718 (local address 127.8.216.190:44877)
0807 19:37:03.115096 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.115103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.125736 (+ 10633us) server_negotiation.cc:407] Connection header received
0807 19:37:03.128798 (+ 3062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.128805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.128890 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.128962 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.134363 (+ 5401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.135449 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.140216 (+ 4767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.140995 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.141103 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:03.151970 (+ 10867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:03.151978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:03.151984 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:03.152042 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:03.156111 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.157809 (+ 1698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.157815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.157818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.157992 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.159239 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.159244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.159247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.159653 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:03.159904 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:03.163220 (+ 3316us) server_negotiation.cc:299] Negotiation successful
0807 19:37:03.163525 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:37:03.167183 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:03.105291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.8.216.188:42835)
0807 19:37:03.105511 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.105518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.122174 (+ 16656us) server_negotiation.cc:407] Connection header received
0807 19:37:03.126055 (+ 3881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.126062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.126162 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.126243 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.131700 (+ 5457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.132726 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.138202 (+ 5476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:03.138954 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:03.139076 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:03.146982 (+ 7906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:03.146989 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:03.146994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:03.147044 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:03.150918 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.157104 (+ 6186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.157110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.157114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.157302 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:03.158475 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:03.158482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:03.158487 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:03.158853 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:03.159137 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:03.166451 (+ 7314us) server_negotiation.cc:299] Negotiation successful
0807 19:37:03.166767 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:37:03.172947 1059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:37:03.187243 980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [term 1 FOLLOWER]: Refusing update from remote peer 4acc680a84a541a5b64eb2f4cf246fca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:03.189015 1047 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:03.196719 902 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [term 1 FOLLOWER]: Refusing update from remote peer 4acc680a84a541a5b64eb2f4cf246fca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:03.200392 1047 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:03.226310 1030 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acc680a84a541a5b64eb2f4cf246fca. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.226691 1030 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:03.232403 1066 mvcc.cc:205] Tried to move back new op lower bound from 7315994309361713152 to 7315994308986359808. Current Snapshot: MvccSnapshot[applied={T|T < 7315994309361713152}]
I20260807 19:37:03.235366 1015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acc680a84a541a5b64eb2f4cf246fca. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.235807 1015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:03.241053 1070 mvcc.cc:205] Tried to move back new op lower bound from 7315994309361713152 to 7315994308986359808. Current Snapshot: MvccSnapshot[applied={T|T < 7315994309361713152}]
I20260807 19:37:03.247246 1030 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.247526 1030 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:03.248765 1015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.249084 1015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:03.249037 1047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.249301 1047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:03.249562 1047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4acc680a84a541a5b64eb2f4cf246fca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4acc680a84a541a5b64eb2f4cf246fca" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 44877 } } peers { permanent_uuid: "ebe18aa97d594856955adde3d52cc034" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 46869 } } peers { permanent_uuid: "860495e3906945b686116e48bb5c89ac" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 42835 } } }
I20260807 19:37:03.249747 1047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:03.255847 1059 catalog_manager.cc:1347] Generated new cluster ID: 80ce0de1891c4dac9f66a44ec8a99a6c
I20260807 19:37:03.255959 1059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:03.279906 1059 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:03.281884 1059 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:03.300223 1059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4acc680a84a541a5b64eb2f4cf246fca: Generated new TSK 0
I20260807 19:37:03.301522 1059 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:37:03.464195 1012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.429170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37890 (local address 127.8.216.190:44877)
0807 19:37:03.429375 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.429382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.459315 (+ 29933us) server_negotiation.cc:407] Connection header received
0807 19:37:03.462219 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.462227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.462361 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.462453 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.463838 (+ 1385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37890: BlockingRecv error: recv got EOF from 127.0.0.1:37890 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 19:37:03.466745 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.431093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.8.216.189:46869)
0807 19:37:03.431312 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.431319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.461861 (+ 30542us) server_negotiation.cc:407] Connection header received
0807 19:37:03.464997 (+ 3136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.465005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.465096 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.465190 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.466463 (+ 1273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49952: BlockingRecv error: recv got EOF from 127.0.0.1:49952 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 19:37:03.469869 1009 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.441368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.8.216.188:42835)
0807 19:37:03.441629 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.441636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.467728 (+ 26092us) server_negotiation.cc:407] Connection header received
0807 19:37:03.468428 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.468435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.468527 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.468605 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.469565 (+ 960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47918: BlockingRecv error: recv got EOF from 127.0.0.1:47918 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 19:37:03.496649 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.494197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.8.216.189:46869)
0807 19:37:03.494449 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.494456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.494664 (+ 208us) server_negotiation.cc:407] Connection header received
0807 19:37:03.495039 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.495047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.495210 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.495341 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.496388 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49956: BlockingRecv error: recv got EOF from 127.0.0.1:49956 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260807 19:37:03.508939 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.506620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49972 (local address 127.8.216.189:46869)
0807 19:37:03.506871 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.506878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.507079 (+ 201us) server_negotiation.cc:407] Connection header received
0807 19:37:03.507410 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.507416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.507524 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.507660 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.508599 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49972: BlockingRecv error: recv got EOF from 127.0.0.1:49972 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 19:37:03.518239 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.515822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.8.216.189:46869)
0807 19:37:03.516104 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.516110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.516451 (+ 341us) server_negotiation.cc:407] Connection header received
0807 19:37:03.516781 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.516787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.516894 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.517024 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.517943 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49978: BlockingRecv error: recv got EOF from 127.0.0.1:49978 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260807 19:37:03.534672 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.527877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49984 (local address 127.8.216.189:46869)
0807 19:37:03.528102 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.528109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.532896 (+ 4787us) server_negotiation.cc:407] Connection header received
0807 19:37:03.533122 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.533129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.533241 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.533377 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.534373 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49984: BlockingRecv error: recv got EOF from 127.0.0.1:49984 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:37:03.548921 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.546370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50000 (local address 127.8.216.189:46869)
0807 19:37:03.546572 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.546579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.547192 (+ 613us) server_negotiation.cc:407] Connection header received
0807 19:37:03.547444 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.547452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.547579 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.547686 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.548644 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50000: BlockingRecv error: recv got EOF from 127.0.0.1:50000 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:37:03.571292 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.568923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.8.216.189:46869)
0807 19:37:03.569144 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.569150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.569695 (+ 545us) server_negotiation.cc:407] Connection header received
0807 19:37:03.569895 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.569901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.570006 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.570125 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.570986 (+ 861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50010: BlockingRecv error: recv got EOF from 127.0.0.1:50010 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 19:37:03.609329 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.607156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.8.216.189:46869)
0807 19:37:03.607374 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.607381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.607744 (+ 363us) server_negotiation.cc:407] Connection header received
0807 19:37:03.607925 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.607931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.608041 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.608226 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.609039 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50018: BlockingRecv error: recv got EOF from 127.0.0.1:50018 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:37:03.679127 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.677016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.8.216.189:46869)
0807 19:37:03.677229 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.677235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.677591 (+ 356us) server_negotiation.cc:407] Connection header received
0807 19:37:03.677759 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.677766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.677910 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.678055 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.678850 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50022: BlockingRecv error: recv got EOF from 127.0.0.1:50022 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 19:37:03.812737 1013 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:03.810415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.8.216.189:46869)
0807 19:37:03.810687 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:03.810693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:03.810999 (+ 306us) server_negotiation.cc:407] Connection header received
0807 19:37:03.811176 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:03.811180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:03.811261 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:03.811514 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:03.812460 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50024: BlockingRecv error: recv got EOF from 127.0.0.1:50024 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:37:03.838551 1027 catalog_manager.cc:1259] Loaded cluster ID: 80ce0de1891c4dac9f66a44ec8a99a6c
I20260807 19:37:03.838667 1027 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: loading cluster ID for follower catalog manager: success
I20260807 19:37:03.841418 1027 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: acquiring CA information for follower catalog manager: success
I20260807 19:37:03.843359 1027 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 860495e3906945b686116e48bb5c89ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:37:03.956212 1042 catalog_manager.cc:1259] Loaded cluster ID: 80ce0de1891c4dac9f66a44ec8a99a6c
I20260807 19:37:03.956329 1042 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: loading cluster ID for follower catalog manager: success
I20260807 19:37:03.958902 1042 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: acquiring CA information for follower catalog manager: success
I20260807 19:37:03.960935 1042 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ebe18aa97d594856955adde3d52cc034: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:37:04.078717 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:04.072567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.8.216.189:46869)
0807 19:37:04.072826 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:04.072835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:04.073086 (+ 251us) server_negotiation.cc:407] Connection header received
0807 19:37:04.073343 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:04.073348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:04.073533 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:04.073672 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:04.074122 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.074881 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.076398 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.076823 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.076996 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:04.077516 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:04.078064 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:04.078247 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:04.078310 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 19:37:04.078403 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:37:04.091596 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:04.085366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.8.216.188:42835)
0807 19:37:04.085914 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:04.085931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:04.085952 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:04.086210 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:04.086214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:04.086388 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:04.086578 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:04.087045 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.087806 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.089243 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.089661 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.089807 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:04.090352 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:04.090933 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:04.091116 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:04.091176 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 19:37:04.091267 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":252,"threads_started":1}
I20260807 19:37:04.109280 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:04.098296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.8.216.190:44877)
0807 19:37:04.101981 (+ 3685us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:04.101989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:04.102613 (+ 624us) server_negotiation.cc:407] Connection header received
0807 19:37:04.102769 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:04.102775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:04.103015 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:04.103236 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:04.103934 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.104917 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.106875 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.107280 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.107440 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:04.108040 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:04.108543 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:04.108671 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:04.108808 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 19:37:04.108905 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3509,"thread_start_us":3336,"threads_started":1}
I20260807 19:37:04.109623 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:04.098455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.8.216.188:42835)
0807 19:37:04.098700 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:04.098709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:04.102777 (+ 4068us) server_negotiation.cc:407] Connection header received
0807 19:37:04.102931 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:04.102938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:04.103022 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:04.103107 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:04.104387 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.105229 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.107384 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.107884 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.108026 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:04.108580 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:04.108724 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:04.108829 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:04.109244 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 19:37:04.109343 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:37:04.111826 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:04.099097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.8.216.189:46869)
0807 19:37:04.099323 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:04.099331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:04.103782 (+ 4451us) server_negotiation.cc:407] Connection header received
0807 19:37:04.104052 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:04.104057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:04.104202 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:04.104308 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:04.106156 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.107244 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.109510 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:04.110214 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:04.110321 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:04.111057 (+ 736us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:04.111203 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:04.111310 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:04.111456 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 19:37:04.111539 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:37:04.115024 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 776
I20260807 19:37:04.116205 845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:04.233285 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 776
I20260807 19:37:04.260345 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 857
I20260807 19:37:04.261600 922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:04.379563 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 857
I20260807 19:37:04.404356 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 935
I20260807 19:37:04.405572 1001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:04.518002 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 935
2026-08-07T19:37:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4982 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:37:04 dist-test-slave-8w9v krb5kdc[1162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:37:04 dist-test-slave-8w9v krb5kdc[1162](info): set up 2 sockets
Aug 07 19:37:04 dist-test-slave-8w9v krb5kdc[1162](info): commencing operation
krb5kdc: starting...
W20260807 19:37:05.707432 9058 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/test-user.keytab.
Aug 07 19:37:05 dist-test-slave-8w9v krb5kdc[1162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:37:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:37:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.216.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
Entry for principal kudu/127.8.216.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab.
I20260807 19:37:06.067787 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45777
--webserver_interface=127.8.216.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:37:06.442623 1175 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:06.443063 1175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:06.443202 1175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:06.443347 1175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:06.451288 1175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:06.451465 1175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:06.451591 1175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:06.451694 1175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:06.451812 1175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:06.462635 1175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.190.keytab
--principal=kudu/127.8.216.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.190:45777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.216.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:06.464094 1175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:06.465555 1175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:06.478665 1182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:06.478718 1185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:06.478807 1181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:06.479555 1175 server_base.cc:1034] running on GCE node
I20260807 19:37:06.480511 1175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:06.482031 1175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:06.483336 1175 hybrid_clock.cc:648] HybridClock initialized: now 1786131426483233 us; error 64 us; skew 500 ppm
I20260807 19:37:06.483722 1175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:06 dist-test-slave-8w9v krb5kdc[1162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131426, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:06.487144 1175 init.cc:377] Logged in from keytab as kudu/127.8.216.190@KRBTEST.COM (short username kudu)
I20260807 19:37:06.488832 1175 webserver.cc:460] Webserver started at http://127.8.216.190:38421/ using document root <none> and password file <none>
I20260807 19:37:06.489584 1175 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:06.489723 1175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:06.490082 1175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:06.493435 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data/instance:
uuid: "359091979928493e9e6c2cf73c156fba"
format_stamp: "Formatted at 2026-08-07 19:37:06 on dist-test-slave-8w9v"
server_key: "692b815d1dcf34f3165072b9d4623279"
server_key_iv: "8b4644999b577e29713b6682bdd4f7b2"
server_key_version: "encryptionkey@0"
I20260807 19:37:06.494274 1175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance:
uuid: "359091979928493e9e6c2cf73c156fba"
format_stamp: "Formatted at 2026-08-07 19:37:06 on dist-test-slave-8w9v"
server_key: "692b815d1dcf34f3165072b9d4623279"
server_key_iv: "8b4644999b577e29713b6682bdd4f7b2"
server_key_version: "encryptionkey@0"
I20260807 19:37:06.500223 1175 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:37:06.504202 1192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:06.505681 1175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:06.505921 1175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal
uuid: "359091979928493e9e6c2cf73c156fba"
format_stamp: "Formatted at 2026-08-07 19:37:06 on dist-test-slave-8w9v"
server_key: "692b815d1dcf34f3165072b9d4623279"
server_key_iv: "8b4644999b577e29713b6682bdd4f7b2"
server_key_version: "encryptionkey@0"
I20260807 19:37:06.506183 1175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:06.523765 1175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:06.528896 1175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:06.529232 1175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:06.549957 1175 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.190:45777
I20260807 19:37:06.549978 1247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.190:45777 every 8 connection(s)
I20260807 19:37:06.551486 1175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/data/info.pb
I20260807 19:37:06.554857 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 1175
I20260807 19:37:06.555071 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-0/wal/instance
I20260807 19:37:06.555668 9058 external_mini_cluster.cc:1428] Setting key 4301ab7737e51ed93c7a5893fe481853
I20260807 19:37:06.555878 1248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:06.566483 1248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:06 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131426, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:37:06.595692 1252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.570721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.8.216.190:45777)
0807 19:37:06.571700 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:06.571716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:06.571922 (+ 206us) server_negotiation.cc:407] Connection header received
0807 19:37:06.575116 (+ 3194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:06.575137 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:06.575415 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:06.575588 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:06.577479 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:06.578699 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:06.581051 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:06.581631 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:06.581768 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:06.586519 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:06.586539 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:06.586562 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:06.586611 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:06.590947 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:06.591986 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:06.591994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:06.591996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:06.592219 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:06.592748 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:06.592752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:06.592754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:06.593298 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:06.593474 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:06.594239 (+ 765us) server_negotiation.cc:299] Negotiation successful
0807 19:37:06.594590 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":759,"spinlock_wait_cycles":13056,"thread_start_us":538,"threads_started":1}
I20260807 19:37:06.595728 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.570721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45777 (local address 127.0.0.1:39604)
0807 19:37:06.571728 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.571778 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:06.574674 (+ 2896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:06.575691 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:06.575702 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:06.576670 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:06.577227 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:06.577256 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:06.578877 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:06.578885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:06.580892 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:06.580920 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:06.581912 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:06.581922 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:06.582152 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:06.582937 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:06.582980 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:06.586192 (+ 3212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:06.591152 (+ 4960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:06.591167 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:06.591185 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:06.591820 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:06.592347 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:06.592355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:06.592361 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:06.592618 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:06.593457 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:06.593467 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:06.593774 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 19:37:06.594107 (+ 333us) client_negotiation.cc:240] Negotiation successful
0807 19:37:06.594774 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":90,"thread_start_us":456,"threads_started":1}
WARNING: no policy specified for kudu/127.8.216.189@KRBTEST.COM; defaulting to no policy
I20260807 19:37:06.598881 1248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.189@KRBTEST.COM" created.
I20260807 19:37:06.600308 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.599797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39276)
0807 19:37:06.599990 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.600169 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:37:06.600701 1194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
I20260807 19:37:06.602331 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.601974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39292)
0807 19:37:06.602120 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.602192 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:37:06.602651 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:39815: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
Entry for principal kudu/127.8.216.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab.
I20260807 19:37:06.639998 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:39815
--webserver_interface=127.8.216.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:37:06.652045 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } attempt: 1
I20260807 19:37:06.653476 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.652868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39296)
0807 19:37:06.653123 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.653291 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:37:06.654700 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.654244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39302)
0807 19:37:06.654418 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.654517 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:37:06.654961 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:39815: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
I20260807 19:37:06.723266 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } attempt: 2
I20260807 19:37:06.724690 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.724016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39308)
0807 19:37:06.724370 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.724490 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 19:37:06.725901 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.725453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39316)
0807 19:37:06.725638 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.725728 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:37:06.726158 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:39815: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
I20260807 19:37:06.817494 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } attempt: 3
I20260807 19:37:06.818994 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.818329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39318)
0807 19:37:06.818620 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.818775 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 19:37:06.820433 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.819846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39320)
0807 19:37:06.820055 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.820208 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:37:06.820732 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:39815: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
I20260807 19:37:06.964066 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } attempt: 4
I20260807 19:37:06.965490 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.964826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39334)
0807 19:37:06.965156 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.965282 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260807 19:37:06.966749 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:06.966315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39344)
0807 19:37:06.966486 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:06.966593 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:37:06.966960 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.189:39815: Network error: Client connection negotiation failed: client connection to 127.8.216.189:39815: connect: Connection refused (error 111)
W20260807 19:37:07.006235 1257 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:07.006644 1257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:07.006765 1257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:07.006908 1257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:07.014896 1257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:07.014997 1257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:07.015100 1257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:07.015188 1257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:07.015305 1257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:07.026016 1257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.189.keytab
--principal=kudu/127.8.216.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.189:39815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.216.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:07.027757 1257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:07.029458 1257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:07.042807 1264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:07.042806 1263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:07.042759 1266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:07.043937 1257 server_base.cc:1034] running on GCE node
I20260807 19:37:07.044962 1257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:07.046490 1257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:07.047717 1257 hybrid_clock.cc:648] HybridClock initialized: now 1786131427047632 us; error 57 us; skew 500 ppm
I20260807 19:37:07.048085 1257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:07.051393 1257 init.cc:377] Logged in from keytab as kudu/127.8.216.189@KRBTEST.COM (short username kudu)
I20260807 19:37:07.053031 1257 webserver.cc:460] Webserver started at http://127.8.216.189:36333/ using document root <none> and password file <none>
I20260807 19:37:07.053766 1257 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:07.053910 1257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:07.054250 1257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:07.057585 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data/instance:
uuid: "e0eb883e2a20412c9682e9c537ff0562"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "7731d64919bce40f65d78eaea1d501c9"
server_key_iv: "474b62d8cabedc81d293ba0ba561701e"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.058432 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal/instance:
uuid: "e0eb883e2a20412c9682e9c537ff0562"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "7731d64919bce40f65d78eaea1d501c9"
server_key_iv: "474b62d8cabedc81d293ba0ba561701e"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.063840 1257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:07.067608 1273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:07.069180 1257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:37:07.069393 1257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal
uuid: "e0eb883e2a20412c9682e9c537ff0562"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "7731d64919bce40f65d78eaea1d501c9"
server_key_iv: "474b62d8cabedc81d293ba0ba561701e"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.069641 1257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:07.100085 1257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:07.104401 1257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:07.104720 1257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:07.124799 1257 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.189:39815
I20260807 19:37:07.124807 1328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.189:39815 every 8 connection(s)
I20260807 19:37:07.126346 1257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/data/info.pb
I20260807 19:37:07.130518 1329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:07.137208 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 1257
I20260807 19:37:07.137421 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-1/wal/instance
I20260807 19:37:07.137895 9058 external_mini_cluster.cc:1428] Setting key 5d1bfc633396ce254ffda4848bff2be3
I20260807 19:37:07.140020 1329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:37:07.166186 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.143057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.8.216.190:45777)
0807 19:37:07.143698 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.143705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.144783 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 19:37:07.147836 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.147841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.147935 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.148039 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.150408 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.151139 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.153371 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.154128 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.154281 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.158364 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.158373 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.158381 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.158445 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.162471 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.163434 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.163440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.163444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.163600 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.164102 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.164106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.164109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.164481 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.164662 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.165476 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.165744 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":333,"threads_started":1}
I20260807 19:37:07.167331 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.143642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45777 (local address 127.0.0.1:39620)
0807 19:37:07.144564 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.144630 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.147464 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.148385 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.148401 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.149285 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.150160 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.150189 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.151325 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.151334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.153193 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.153207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.154374 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.154379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.154600 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.155376 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.155407 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.158051 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.162728 (+ 4677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.162736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.162750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.163253 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.163733 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.163738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.163743 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.163952 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.164631 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.164640 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.165062 (+ 422us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.165370 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.166002 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":374,"threads_started":1}
I20260807 19:37:07.169520 1329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.216.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.216.188@KRBTEST.COM" created.
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:07.186031 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.170499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39346)
0807 19:37:07.170752 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.170781 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.170938 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.171959 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.171965 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.172552 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.172772 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.172786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.174624 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.174628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.175625 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.175637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.176688 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.176693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.176913 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.177660 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.177677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.179867 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.183385 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.183391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.183398 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.183763 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.184170 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.184176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.184179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.184324 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.184952 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.184961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.185137 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.185298 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.185607 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:37:07.186123 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.170709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.8.216.189:39815)
0807 19:37:07.171476 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.171483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.171511 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:37:07.171561 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.171567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.171767 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.171889 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.172964 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.174460 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.175762 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.176515 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.176621 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.180104 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.180156 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.180182 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.180220 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.183241 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.183905 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.183919 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.183923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.184033 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.184441 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.184446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.184449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.184813 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.185102 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.185462 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.185728 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":597,"thread_start_us":438,"threads_started":1}
I20260807 19:37:07.189172 1329 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:37:07.190613 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.190115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43274)
0807 19:37:07.190301 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.190446 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:37:07.190943 1277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.192880 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.192299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43284)
0807 19:37:07.192519 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.192681 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:37:07.193284 1329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
Entry for principal kudu/127.8.216.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab.
I20260807 19:37:07.219920 9058 external_mini_cluster.cc:1326] Running /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
/tmp/dist-test-task0Yp7oG/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:36991
--webserver_interface=127.8.216.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.216.148:46133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:37:07.242715 1329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 1
I20260807 19:37:07.244170 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.243492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43290)
0807 19:37:07.243769 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.243922 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 19:37:07.245494 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.244990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43296)
0807 19:37:07.245207 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.245318 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:37:07.245740 1329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.266244 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } attempt: 5
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131426, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:07.287678 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.267318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39358 (local address 127.8.216.189:39815)
0807 19:37:07.267522 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.267527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.267551 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:37:07.267764 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.267769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.267847 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.267951 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.269039 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.269742 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.271371 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.271802 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.271899 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.281170 (+ 9271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.281177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.281180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.281228 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.284494 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.285369 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.285384 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.285389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.285533 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.285998 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.286004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.286010 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.286348 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.286542 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.286999 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.287301 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:37:07.287685 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.267157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39358)
0807 19:37:07.267402 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.267425 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.267573 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.268016 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.268021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.268596 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.268850 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.268864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.269885 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.269890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.271217 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.271233 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.271960 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.271969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.272237 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.277501 (+ 5264us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.277539 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.280833 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.284706 (+ 3873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.284713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.284718 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.285202 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.285681 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.285687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.285691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.285858 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.286494 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.286499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.286687 (+ 188us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.286853 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.287294 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:37:07.288931 1248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:37:07.290158 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.289698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43304)
0807 19:37:07.289897 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.290001 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:37:07.291208 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.290856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43310)
0807 19:37:07.291007 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.291097 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:37:07.291529 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.314003 1329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 2
I20260807 19:37:07.315474 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.314861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43320)
0807 19:37:07.315117 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.315242 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 19:37:07.317109 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.316575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43334)
0807 19:37:07.316789 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.316908 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 19:37:07.317452 1329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.342849 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 1
I20260807 19:37:07.344547 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.343772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43348)
0807 19:37:07.344067 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.344300 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 19:37:07.346230 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.345752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43356)
0807 19:37:07.345978 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.346075 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 19:37:07.346529 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.408787 1329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 3
I20260807 19:37:07.410421 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.409702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43358)
0807 19:37:07.409998 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.410160 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 19:37:07.411861 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.411249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43368)
0807 19:37:07.411617 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.411735 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247}
W20260807 19:37:07.412160 1329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.414817 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 2
I20260807 19:37:07.416289 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.415599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43378)
0807 19:37:07.415890 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.416003 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:37:07.417806 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.417265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43388)
0807 19:37:07.417526 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.417634 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260807 19:37:07.418103 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.524348 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 3
I20260807 19:37:07.525800 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.525150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43390)
0807 19:37:07.525418 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.525551 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:37:07.527550 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.526816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43402)
0807 19:37:07.527092 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.527199 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260807 19:37:07.527966 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.555457 1329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 4
I20260807 19:37:07.556825 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.556291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43404)
0807 19:37:07.556563 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.556655 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:37:07.557925 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.557550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43410)
0807 19:37:07.557714 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.557805 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:37:07.558184 1329 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
W20260807 19:37:07.655736 1337 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:37:07.656219 1337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:37:07.656322 1337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:37:07.656456 1337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:37:07.664548 1337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:37:07.664670 1337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:37:07.664775 1337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:37:07.664872 1337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:37:07.665002 1337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:37:07.676734 1337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.216.148:46133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.216.190:45777,127.8.216.189:39815,127.8.216.188:36991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/krb5kdc/kudu_127.8.216.188.keytab
--principal=kudu/127.8.216.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.216.188:36991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.216.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:32:26 UTC on ccycloud.quasar-asbpqd.root.comops.site
build id 4573430
ASAN enabled
I20260807 19:37:07.678242 1337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:37:07.679749 1337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:37:07.694411 1342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:07.694334 1343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:37:07.694446 1345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:37:07.695546 1337 server_base.cc:1034] running on GCE node
I20260807 19:37:07.696580 1337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:37:07.698215 1337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:37:07.699505 1337 hybrid_clock.cc:648] HybridClock initialized: now 1786131427699430 us; error 72 us; skew 500 ppm
I20260807 19:37:07.699867 1337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:37:07.703398 1337 init.cc:377] Logged in from keytab as kudu/127.8.216.188@KRBTEST.COM (short username kudu)
I20260807 19:37:07.705251 1337 webserver.cc:460] Webserver started at http://127.8.216.188:38575/ using document root <none> and password file <none>
I20260807 19:37:07.705232 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 4
I20260807 19:37:07.706269 1337 fs_manager.cc:359] Metadata directory not provided
I20260807 19:37:07.706408 1337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:37:07.706526 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.706011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43414)
0807 19:37:07.706221 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.706320 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:37:07.706835 1337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:37:07.707731 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.707361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43426)
0807 19:37:07.707514 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.707597 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:37:07.707985 1248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.216.188:36991: Network error: Client connection negotiation failed: client connection to 127.8.216.188:36991: connect: Connection refused (error 111)
I20260807 19:37:07.710222 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data/instance:
uuid: "4baf9f61694d403a9d211313917b10c1"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "d28ea3a8f26eb2f6fff324980bd5bb6a"
server_key_iv: "a9bfd47151e4690fef43d39dde9e9159"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.711203 1337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal/instance:
uuid: "4baf9f61694d403a9d211313917b10c1"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "d28ea3a8f26eb2f6fff324980bd5bb6a"
server_key_iv: "a9bfd47151e4690fef43d39dde9e9159"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.717152 1337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:37:07.721459 1353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:37:07.723203 1337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:37:07.723577 1337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data,/tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal
uuid: "4baf9f61694d403a9d211313917b10c1"
format_stamp: "Formatted at 2026-08-07 19:37:07 on dist-test-slave-8w9v"
server_key: "d28ea3a8f26eb2f6fff324980bd5bb6a"
server_key_iv: "a9bfd47151e4690fef43d39dde9e9159"
server_key_version: "encryptionkey@0"
I20260807 19:37:07.723878 1337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:37:07.741835 1337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:37:07.748263 1337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:37:07.748656 1337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:37:07.772403 1408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.216.188:36991 every 8 connection(s)
I20260807 19:37:07.772396 1337 rpc_server.cc:307] RPC server started. Bound to: 127.8.216.188:36991
I20260807 19:37:07.774484 1337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/data/info.pb
I20260807 19:37:07.779392 9058 external_mini_cluster.cc:1388] Started /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu as pid 1337
I20260807 19:37:07.779659 9058 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task0Yp7oG/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131039239543-9058-0/minicluster-data/master-2/wal/instance
I20260807 19:37:07.779665 1409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:37:07.780359 9058 external_mini_cluster.cc:1428] Setting key f8a48982d84498dcd5d90eb221ff9140
I20260807 19:37:07.791302 1409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
I20260807 19:37:07.809310 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.782528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.8.216.188:36991)
0807 19:37:07.783447 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.783459 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.783509 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:37:07.785492 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.785525 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.786005 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.786419 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.788596 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.789985 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.791955 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.792946 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.793115 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.798476 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.798488 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.798507 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.798566 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.803290 (+ 4724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.804338 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.804346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.804354 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.804579 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.805110 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.805115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.805119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.806022 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.806292 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.807059 (+ 767us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.807576 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":425,"threads_started":1}
I20260807 19:37:07.814888 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.794784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.190:45777 (local address 127.0.0.1:39634)
0807 19:37:07.795669 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.795719 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.795892 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.796791 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.796798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.797566 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.797884 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.797898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.799615 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.799622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.801209 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.801224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.802373 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.802378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.802528 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.803316 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.803341 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.806515 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.811180 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.811191 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.811198 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.811924 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.812553 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.812557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.812560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.812726 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.813583 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.813593 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.813874 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.814125 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.814469 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":434,"threads_started":1}
I20260807 19:37:07.816112 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.794843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.8.216.190:45777)
0807 19:37:07.796417 (+ 1574us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.796422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.796438 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:37:07.796493 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.796496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.796571 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.796711 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.798121 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.799370 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.801441 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.802100 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.802271 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.806818 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.806826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.806830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.806886 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.810898 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.812101 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.812106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.812108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.812361 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.812908 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.812912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.812914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.813291 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.813485 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.815310 (+ 1825us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.815604 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1461,"thread_start_us":1318,"threads_started":1}
I20260807 19:37:07.817816 1409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.190@KRBTEST.COM
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:07.834091 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.813724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.8.216.190:45777)
0807 19:37:07.814408 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.814413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.814434 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:07.814477 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.814480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.814539 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.814631 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.816600 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.817951 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.819722 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.820522 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.820632 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.826287 (+ 5655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.826296 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.826301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.826347 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.830391 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.831540 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.831545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.831550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.831731 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.832318 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.832323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.832325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.832656 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.832976 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.833362 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.833626 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":338,"threads_started":1}
I20260807 19:37:07.836021 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.818816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.189:39815 (local address 127.0.0.1:39362)
0807 19:37:07.819000 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.819028 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.819156 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.819973 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.819977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.820535 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.820750 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.820763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.822087 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.822093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.823757 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.823774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.824736 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.824741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.824887 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.825511 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.825530 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.828882 (+ 3352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.832927 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.832933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.832937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.833535 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.834074 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.834077 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.834080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.834342 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.834997 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.835002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.835199 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.835404 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.835705 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:37:07.836283 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.818796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.8.216.189:39815)
0807 19:37:07.819416 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.819421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.819436 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:37:07.819615 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.819619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.819750 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.819897 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.820962 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.821913 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.823952 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.824542 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.824688 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.829147 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.829153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.829156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.829194 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.832722 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.833714 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.833720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.833724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.833933 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.834496 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.834501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.834504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.834856 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.834985 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.835559 (+ 574us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.835819 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":355,"threads_started":1}
I20260807 19:37:07.837450 1409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.188@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.216.189@KRBTEST.COM
I20260807 19:37:07.852350 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.838521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.8.216.188:36991)
0807 19:37:07.838763 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.838769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.838868 (+ 99us) server_negotiation.cc:407] Connection header received
0807 19:37:07.839039 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.839044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.839114 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.839222 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.839965 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.840735 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.841900 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.842348 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.842498 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.846475 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.846480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.846483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.846515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.849455 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.850157 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.850165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.850168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.850297 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.850657 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.850665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.850668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.850986 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.851152 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.851516 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.851992 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":206,"server-negotiator.queue_time_us":158}
I20260807 19:37:07.852350 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.838504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43452)
0807 19:37:07.838745 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.838789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.838931 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.839237 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.839241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.839685 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.839823 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.839832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.840864 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.840872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.841736 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.841747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.842519 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.842525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.842746 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.843508 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.843532 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.846213 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.849613 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.849619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.849623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.850028 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.850410 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.850414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.850418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.850549 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.851109 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.851114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.851266 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.851436 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.851822 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1}
I20260807 19:37:07.854079 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.836556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.8.216.189:39815)
0807 19:37:07.836807 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.836815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.836836 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:07.836900 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.836908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.837000 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.837099 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.838912 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.839668 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.841122 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.841539 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.841701 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.847161 (+ 5460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.847171 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.847177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.847230 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.850216 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.851046 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.851057 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.851059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.851204 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.851734 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.851738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.851740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.852041 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.852373 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.853471 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.853746 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":568,"threads_started":1}
I20260807 19:37:07.857460 1329 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 5
I20260807 19:37:07.859261 1409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: Bootstrap starting.
I20260807 19:37:07.860667 9058 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:37:07.863848 1409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:07.865396 1409 log.cc:824] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131427, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.189@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
I20260807 19:37:07.864183 1221 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "359091979928493e9e6c2cf73c156fba"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39644
I20260807 19:37:07.869266 1409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: No bootstrap required, opened a new log
I20260807 19:37:07.875597 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.858777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.8.216.188:36991)
0807 19:37:07.859137 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.859144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.859165 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:37:07.859221 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.859226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.859312 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.859427 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.860604 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.861483 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.862792 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.863355 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.863504 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.867853 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.867863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.867871 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.867914 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.871972 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.873039 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.873047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.873052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.873265 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.873846 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.873852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.873855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.874330 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:07.874527 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:07.875089 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 19:37:07.875299 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:37:07.875635 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.858203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43462)
0807 19:37:07.858430 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.858457 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.858695 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.859461 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.859466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.860028 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.860304 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.860317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.861656 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.861665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.862636 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.862649 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.863539 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.863548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.863883 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.864686 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.864710 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.867604 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.872231 (+ 4627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.872239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.872244 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.872885 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.873425 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.873431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.873434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.873688 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:07.874467 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:07.874472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:07.874661 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 19:37:07.874838 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:37:07.875197 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:37:07.881232 1409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:07.881803 1409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4baf9f61694d403a9d211313917b10c1, State: Initialized, Role: FOLLOWER
I20260807 19:37:07.882390 1409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:07.882699 1329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: Bootstrap starting.
I20260807 19:37:07.883558 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:07.884105 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:07.885082 1409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:07.887784 1329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:07.889760 1329 log.cc:824] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:07.894443 1329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: No bootstrap required, opened a new log
W20260807 19:37:07.902280 1433 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:37:07.902520 1433 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:37:07.907013 1329 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:07.907588 1329 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0eb883e2a20412c9682e9c537ff0562, State: Initialized, Role: FOLLOWER
I20260807 19:37:07.908288 1329 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:07.909919 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:07.910995 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:07.911129 1329 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:07.899976 1337 master_runner.cc:186] Error getting master registration for 127.8.216.190:45777: OK, instance_id { permanent_uuid: "359091979928493e9e6c2cf73c156fba" instance_seqno: 1786131426544012 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:37:07.912676 1337 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 19:37:07.923400 1446 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:37:07.923633 1446 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:37:07.919849 1257 master_runner.cc:186] Error getting master registration for 127.8.216.190:45777: OK, instance_id { permanent_uuid: "359091979928493e9e6c2cf73c156fba" instance_seqno: 1786131426544012 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:37:07.934142 1257 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:37:07.985250 1248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } attempt: 5
Aug 07 19:37:07 dist-test-slave-8w9v krb5kdc[1162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131426, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.216.190@KRBTEST.COM for kudu/127.8.216.188@KRBTEST.COM
W20260807 19:37:08.001703 1360 tablet.cc:2369] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:08.002146 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.986136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.8.216.188:36991)
0807 19:37:07.986338 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:07.986344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:07.986728 (+ 384us) server_negotiation.cc:407] Connection header received
0807 19:37:07.986813 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:07.986817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:07.986886 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:07.986998 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:07.988026 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.988865 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.990784 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.991318 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.991467 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.995792 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:07.995797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:07.995800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:07.995837 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:07.998897 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:07.999795 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:07.999800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:07.999803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:07.999947 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.000463 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.000468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.000470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.000777 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:08.000946 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:08.001364 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 19:37:08.001639 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:37:08.003425 1250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:07.986061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.216.188:36991 (local address 127.0.0.1:43466)
0807 19:37:07.986394 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 19:37:07.986412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:37:07.986584 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:37:07.987048 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:37:07.987053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:37:07.987618 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:37:07.987855 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.987867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.989017 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.989023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.990611 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:37:07.990630 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:37:07.991488 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:37:07.991494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:37:07.991737 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:07.992512 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:37:07.992542 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:37:07.995549 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:37:07.999070 (+ 3521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:07.999077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:07.999081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:07.999647 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:08.000077 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:37:08.000085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:37:08.000088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:37:08.000332 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:37:08.000907 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:37:08.000912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:37:08.001099 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 19:37:08.001253 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 19:37:08.003003 (+ 1750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 19:37:08.009846 1248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba: Bootstrap starting.
I20260807 19:37:08.014384 1248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba: Neither blocks nor log segments found. Creating new log.
I20260807 19:37:08.015858 1248 log.cc:824] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba: Log is configured to *not* fsync() on all Append() calls
I20260807 19:37:08.019989 1248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba: No bootstrap required, opened a new log
I20260807 19:37:08.028074 1248 raft_consensus.cc:348] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:08.028774 1248 raft_consensus.cc:729] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359091979928493e9e6c2cf73c156fba, State: Initialized, Role: FOLLOWER
I20260807 19:37:08.029366 1248 consensus_queue.cc:260] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:08.036756 1449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.037204 1449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:08.039184 1248 sys_catalog.cc:564] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:37:08.039271 1221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:37:08.039572 1221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:08.044154 1221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:08.047581 1303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "359091979928493e9e6c2cf73c156fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e0eb883e2a20412c9682e9c537ff0562"
I20260807 19:37:08.047691 1384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "359091979928493e9e6c2cf73c156fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4baf9f61694d403a9d211313917b10c1"
I20260807 19:37:08.048205 1384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:08.048249 1303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:37:08.052037 1303 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359091979928493e9e6c2cf73c156fba in term 1.
I20260807 19:37:08.052037 1384 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 359091979928493e9e6c2cf73c156fba in term 1.
I20260807 19:37:08.052884 1194 leader_election.cc:302] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 359091979928493e9e6c2cf73c156fba, 4baf9f61694d403a9d211313917b10c1, e0eb883e2a20412c9682e9c537ff0562; no voters:
I20260807 19:37:08.053283 1221 leader_election.cc:288] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [CANDIDATE]: Term 1 election: Requested vote from peers e0eb883e2a20412c9682e9c537ff0562 (127.8.216.189:39815), 4baf9f61694d403a9d211313917b10c1 (127.8.216.188:36991)
I20260807 19:37:08.053431 1449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:37:08.065325 1449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [term 1 LEADER]: Becoming Leader. State: Replica: 359091979928493e9e6c2cf73c156fba, State: Running, Role: LEADER
I20260807 19:37:08.066270 1449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } }
I20260807 19:37:08.077926 1451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359091979928493e9e6c2cf73c156fba. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.078667 1451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:08.086737 1458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:37:08.106172 1458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 19:37:08.107702 1280 tablet.cc:2369] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:37:08.129138 1472 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:37:08.134788 1303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [term 1 FOLLOWER]: Refusing update from remote peer 359091979928493e9e6c2cf73c156fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:08.136710 1384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [term 1 FOLLOWER]: Refusing update from remote peer 359091979928493e9e6c2cf73c156fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:37:08.138115 1449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:08.138896 1449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:37:08.175248 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359091979928493e9e6c2cf73c156fba. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.175637 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:08.183688 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:08.124852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.8.216.188:36991)
0807 19:37:08.125160 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.125168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.139597 (+ 14429us) server_negotiation.cc:407] Connection header received
0807 19:37:08.139774 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.139780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.139887 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.140012 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.142293 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.143448 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.145426 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.146123 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.146224 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:08.160312 (+ 14088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:08.160321 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:08.160326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:08.160385 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:08.165366 (+ 4981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.177991 (+ 12625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.177998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.178002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.178193 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.178736 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.178742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.178745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.179225 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:08.179500 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:08.182926 (+ 3426us) server_negotiation.cc:299] Negotiation successful
0807 19:37:08.183258 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 19:37:08.188099 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:08.102764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.8.216.190:45777)
0807 19:37:08.103025 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.103033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.126942 (+ 23909us) server_negotiation.cc:407] Connection header received
0807 19:37:08.128003 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.128009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.128112 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.128217 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.133493 (+ 5276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.134677 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.152667 (+ 17990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.153425 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.153542 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:08.166757 (+ 13215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:08.166766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:08.166772 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:08.166824 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:08.171131 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.180165 (+ 9034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.180173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.180177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.180355 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.181580 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.181586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.181589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.182016 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:08.182283 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:08.187361 (+ 5078us) server_negotiation.cc:299] Negotiation successful
0807 19:37:08.187669 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:37:08.188706 1449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.189061 1449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:08.190012 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:08.109421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.8.216.189:39815)
0807 19:37:08.109630 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.109636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.129069 (+ 19433us) server_negotiation.cc:407] Connection header received
0807 19:37:08.129242 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.129248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.129349 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.129487 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.135501 (+ 6014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.136666 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.150728 (+ 14062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:08.151420 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:08.151533 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:08.172403 (+ 20870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:37:08.172412 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:37:08.172417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:37:08.172475 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:37:08.176912 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.180892 (+ 3980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.180898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.180901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.181078 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:37:08.185674 (+ 4596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:37:08.185681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:37:08.185685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:37:08.186245 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:37:08.186589 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:08.189310 (+ 2721us) server_negotiation.cc:299] Negotiation successful
0807 19:37:08.189605 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:37:08.193998 1458 catalog_manager.cc:1347] Generated new cluster ID: 78585062181b430d87962091d623cb5e
I20260807 19:37:08.194140 1458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:37:08.194523 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.194801 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:08.195152 1449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.195479 1449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba [sys.catalog]: This master's current role is: LEADER
I20260807 19:37:08.196628 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 359091979928493e9e6c2cf73c156fba. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.197024 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:08.210546 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "359091979928493e9e6c2cf73c156fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "359091979928493e9e6c2cf73c156fba" member_type: VOTER last_known_addr { host: "127.8.216.190" port: 45777 } } peers { permanent_uuid: "e0eb883e2a20412c9682e9c537ff0562" member_type: VOTER last_known_addr { host: "127.8.216.189" port: 39815 } } peers { permanent_uuid: "4baf9f61694d403a9d211313917b10c1" member_type: VOTER last_known_addr { host: "127.8.216.188" port: 36991 } } }
I20260807 19:37:08.210884 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:37:08.230195 1458 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:37:08.232239 1458 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:37:08.256723 1458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 359091979928493e9e6c2cf73c156fba: Generated new TSK 0
I20260807 19:37:08.258056 1458 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:37:08.413489 1416 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.385795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.8.216.190:45777)
0807 19:37:08.386057 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.386067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.409099 (+ 23032us) server_negotiation.cc:407] Connection header received
0807 19:37:08.411651 (+ 2552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.411658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.411763 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.411843 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.413165 (+ 1322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39662: BlockingRecv error: recv got EOF from 127.0.0.1:39662 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 19:37:08.413965 1412 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.406615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43486 (local address 127.8.216.188:36991)
0807 19:37:08.406849 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.406856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.411601 (+ 4745us) server_negotiation.cc:407] Connection header received
0807 19:37:08.412297 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.412305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.412406 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.412483 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.413639 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43486: BlockingRecv error: recv got EOF from 127.0.0.1:43486 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 19:37:08.415669 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.392162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.8.216.189:39815)
0807 19:37:08.392382 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.392391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.411326 (+ 18935us) server_negotiation.cc:407] Connection header received
0807 19:37:08.414112 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.414118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.414207 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.414278 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.415411 (+ 1133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39398: BlockingRecv error: recv got EOF from 127.0.0.1:39398 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 19:37:08.462478 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.449784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.8.216.189:39815)
0807 19:37:08.449982 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.449989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.459769 (+ 9780us) server_negotiation.cc:407] Connection header received
0807 19:37:08.460376 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.460382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.460529 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.460611 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.462046 (+ 1435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39404: BlockingRecv error: recv got EOF from 127.0.0.1:39404 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:37:08.510932 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.497855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.8.216.189:39815)
0807 19:37:08.498054 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.498060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.508817 (+ 10757us) server_negotiation.cc:407] Connection header received
0807 19:37:08.509150 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.509159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.509303 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.509394 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.510668 (+ 1274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39410: BlockingRecv error: recv got EOF from 127.0.0.1:39410 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:37:08.545392 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.542711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.8.216.189:39815)
0807 19:37:08.542905 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.542912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.543487 (+ 575us) server_negotiation.cc:407] Connection header received
0807 19:37:08.543700 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.543707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.543837 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.543985 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.545047 (+ 1062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39426: BlockingRecv error: recv got EOF from 127.0.0.1:39426 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:37:08.581892 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.579682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.8.216.189:39815)
0807 19:37:08.580010 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.580017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.580089 (+ 72us) server_negotiation.cc:407] Connection header received
0807 19:37:08.580443 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.580449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.580559 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.580689 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.581638 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39438: BlockingRecv error: recv got EOF from 127.0.0.1:39438 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
W20260807 19:37:08.596783 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.594475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.8.216.189:39815)
0807 19:37:08.594722 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.594731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.595025 (+ 294us) server_negotiation.cc:407] Connection header received
0807 19:37:08.595337 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.595345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.595444 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.595565 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.596514 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39450: BlockingRecv error: recv got EOF from 127.0.0.1:39450 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 19:37:08.618777 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.616622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.8.216.189:39815)
0807 19:37:08.616833 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.616842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.617298 (+ 456us) server_negotiation.cc:407] Connection header received
0807 19:37:08.617538 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.617544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.617631 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.617748 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.618519 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39452: BlockingRecv error: recv got EOF from 127.0.0.1:39452 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 19:37:08.657797 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.655430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.8.216.189:39815)
0807 19:37:08.655771 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.655779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.656106 (+ 327us) server_negotiation.cc:407] Connection header received
0807 19:37:08.656413 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.656417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.656493 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.656587 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.657535 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39458: BlockingRecv error: recv got EOF from 127.0.0.1:39458 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 19:37:08.727684 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.725699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.8.216.189:39815)
0807 19:37:08.725892 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.725898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.726270 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:37:08.726442 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.726447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.726540 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.726662 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.727448 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39472: BlockingRecv error: recv got EOF from 127.0.0.1:39472 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:37:08.863931 1417 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:37:08.861962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.8.216.189:39815)
0807 19:37:08.862168 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:08.862173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:08.862583 (+ 410us) server_negotiation.cc:407] Connection header received
0807 19:37:08.862764 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:08.862769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:08.862845 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:08.862949 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:37:08.863729 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39486: BlockingRecv error: recv got EOF from 127.0.0.1:39486 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:37:08.904718 1433 catalog_manager.cc:1259] Loaded cluster ID: 78585062181b430d87962091d623cb5e
I20260807 19:37:08.904829 1433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: loading cluster ID for follower catalog manager: success
I20260807 19:37:08.907411 1433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: acquiring CA information for follower catalog manager: success
I20260807 19:37:08.909402 1433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4baf9f61694d403a9d211313917b10c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:37:08.925913 1446 catalog_manager.cc:1259] Loaded cluster ID: 78585062181b430d87962091d623cb5e
I20260807 19:37:08.926062 1446 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: loading cluster ID for follower catalog manager: success
I20260807 19:37:08.929781 1446 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: acquiring CA information for follower catalog manager: success
I20260807 19:37:08.931815 1446 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0eb883e2a20412c9682e9c537ff0562: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:37:09.131693 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:09.124676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.8.216.189:39815)
0807 19:37:09.125489 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:09.125494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:09.125510 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:37:09.125574 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:09.125577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:09.125751 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:09.125889 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:09.126403 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.127145 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.128849 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.129544 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.129705 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:09.130526 (+ 821us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:09.131086 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:09.131216 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:09.131328 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:37:09.131405 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700}
I20260807 19:37:09.146031 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:09.137776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.8.216.188:36991)
0807 19:37:09.140505 (+ 2729us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:09.140510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:09.140534 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:37:09.140638 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:09.140642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:09.140936 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:09.141124 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:09.141560 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.142301 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.143644 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.144061 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.144245 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:09.144776 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:09.145370 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:09.145538 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:09.145600 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 19:37:09.145689 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2615,"thread_start_us":2513,"threads_started":1}
I20260807 19:37:09.161088 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:09.152263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39502 (local address 127.8.216.189:39815)
0807 19:37:09.152509 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:09.152518 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:09.154270 (+ 1752us) server_negotiation.cc:407] Connection header received
0807 19:37:09.154578 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:09.154582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:09.154652 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:09.154758 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:09.155744 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.156598 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.158929 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.159623 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.159781 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:09.160300 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:09.160455 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:09.160553 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:09.160698 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 19:37:09.160780 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:37:09.161113 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:09.153342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.8.216.188:36991)
0807 19:37:09.153567 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:09.153576 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:09.154533 (+ 957us) server_negotiation.cc:407] Connection header received
0807 19:37:09.154803 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:09.154810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:09.154901 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:09.154995 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:09.155793 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.156885 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.158787 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.159452 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.159591 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:09.160202 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:09.160353 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:09.160421 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:09.160655 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 19:37:09.160741 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:37:09.161931 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:37:09.152222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39672 (local address 127.8.216.190:45777)
0807 19:37:09.152993 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0807 19:37:09.153002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:37:09.153771 (+ 769us) server_negotiation.cc:407] Connection header received
0807 19:37:09.154214 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:37:09.154222 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:37:09.154476 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:37:09.154647 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:37:09.155623 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.156886 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.159188 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:37:09.159669 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:37:09.159800 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:37:09.160521 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:37:09.161014 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:37:09.161146 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:37:09.161568 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 19:37:09.161649 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":438,"threads_started":1}
I20260807 19:37:09.166734 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 1175
I20260807 19:37:09.168340 1242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:09.283684 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 1175
I20260807 19:37:09.310346 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 1257
I20260807 19:37:09.311610 1324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:09.424613 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 1257
I20260807 19:37:09.450227 9058 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu pid 1337
I20260807 19:37:09.451392 1404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:37:09.563617 9058 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task0Yp7oG/build/asan/bin/kudu with pid 1337
2026-08-07T19:37:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5044 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10028 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (390290 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST